JP2015173923A - Image processing device, image processing method, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image processing device which can calculate a direction of a subject without depending on the subject in a photographed image.SOLUTION: An image processing device has: computing means which computes to determine the direction of a subject in image data on the basis of a value showing tendency after comparing intensity distribution of a pixel value for each of a plurality of regions in the image data of the subject among a plurality of regions and a value defined every subject in advance so as to associate the value showing the tendency with the direction of the subject in the image data; and display processing means which performs processing to allow display means to display on the basis of a computation result computed by the computing means an image of the subject so that the subject in the image of the subject faces in a prescribed direction with respect to the display means displaying the image of the subject on the basis of the image data.

Description

  The present invention relates to an image processing apparatus that performs processing for calculating the orientation of a subject in a captured image, and an image processing method and program thereof.

  When digital image processing is performed on a radiographic image generated by a radiation sensor, the processing may be performed in consideration of an anatomical structure including a subject direction in the captured image. In addition, it is desired from the user that the image is automatically rotated and displayed in a direction in which the image can be easily observed at a specified angle. In order to perform these digital image processing and automatic image rotation, it is necessary to determine the orientation of the subject in the image. Patent Document 1 proposes a technique for extracting a vertebral body region from an anatomical feature of a vertebral body and discriminating the orientation of the vertebral body for a radiographic image of the chest.

JP 2009-201872 A

  However, the method proposed in Patent Document 1 is a technique for extracting an anatomical feature of a vertebral body from a vertebral body region in a chest image, and other parts and machines such as the head and limbs. It cannot be said that it corresponds to every subject such as an inspection object which is a target of a typical nondestructive inspection. Therefore, it is difficult to apply the invention described in Patent Document 1 to various subjects.

  Therefore, an object of the present invention is to provide an image processing apparatus capable of calculating the orientation of a subject in a captured image regardless of the subject.

Therefore, the image processing apparatus of the present invention is
In order to associate a value indicating the tendency of comparing the intensity distribution of pixel values for each of a plurality of regions in the image data of the subject between the plurality of regions, the value indicating the tendency and the orientation of the subject in the image data in advance. Based on the value determined for each sample, a calculation unit that performs a calculation to determine the orientation of the subject in the image data, and based on the calculation result calculated by the calculation unit, based on the image data Display processing means for performing processing for displaying the image of the subject on the display means so that the subject in the image of the subject is in a predetermined orientation with respect to the display means for displaying the image of the subject; It is characterized by having.

  The present invention can provide an image processing apparatus capable of calculating the orientation of a subject in a captured image regardless of the subject.

It is a figure explaining the structure of the image processing apparatus concerning embodiment. It is the figure which showed an example of the direction defined based on the reference | standard direction of the image processing apparatus concerning embodiment, and object direction. 3 is a flowchart illustrating an operation of the image processing apparatus according to the embodiment. 3 is a flowchart illustrating an operation of extracting feature values of the image processing apparatus according to the embodiment. It is the figure which showed an example of the picked-up image made into embodiment, and the edge strength image data extracted from a picked-up image. It is the figure which showed an example of edge strength image data and area | region edge image data. It is the figure which showed an example of the picked-up image made into embodiment, and the direction edge image extracted from a picked-up image.

  Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the drawings. First, an image processing apparatus 200 according to an embodiment of the present invention will be described with reference to FIG. An image processing apparatus 200 according to an embodiment of the present invention is applied to, for example, a radiation imaging apparatus 100 as shown in FIG. That is, the radiation imaging apparatus 100 is a radiation imaging apparatus including an image processing apparatus that performs an operation for determining the orientation of the subject. The image processing apparatus 200 according to the present embodiment will be described below with reference to the drawings. The radiation imaging apparatus 100 includes a radiation generation unit 101 and a radiation sensor 104. The radiation generation unit 101 irradiates the subject 102 with radiation 103. The radiation sensor 104 receives the radiation 103 transmitted through the subject 102 and generates radiation image data. That is, the radiation sensor 104 functions as an imaging unit that generates captured image data of the subject 102 irradiated with radiation. The captured image data of the subject is transmitted / received via the bus 115 which is a communication means. In the present embodiment, it is assumed that electromagnetic waves such as X-rays and γ rays, α rays, and β rays are included in the radiation.

  Further, the radiation imaging apparatus 100 includes a preprocessing unit 105, an operation unit 106, a storage unit 107, and a calculation unit 108. The preprocessing unit 105 performs processing for correcting the characteristics of the radiation sensor 104 on the captured image data output from the radiation sensor 104. The operation unit 106 inputs a user operation instruction and gives the operation instruction to each unit. The storage unit 107 holds at least information on the subject of the patient to be imaged and a value predetermined for each subject. The storage unit 107 may further hold image processing parameters and image correction data. The calculation unit 108 performs calculation for determining the orientation of the subject in the radiation image data. In other words, the calculation unit 108 is based on a value predetermined for each subject in order to associate a value indicating the distribution tendency of the change amount of the pixel value for each of the plurality of regions in the image data of the subject with the orientation of the subject. To calculate the orientation of the subject in the image data. The “pixel value” is a luminance value or density value of a pixel. The “pixel value change amount” is a difference between a luminance value and a density value which are pixel values of a pixel of interest and one or a plurality of adjacent pixels. The “pixel value intensity distribution” indicates a distribution of pixel value intensity in image data, and may be standardized data. For example, it refers to an edge intensity image or an edge intensity distribution. In addition, the “subject” is a subject to be examined and refers to a person, an object, or an animal to be examined. For example, a chest part, a lung field, an abdomen, an extremity, a head, etc. which are parts of a human body are shown. Further, the “value determined in advance for each subject” is a value for associating the feature amount and the direction of the subject in order to calculate the direction of the subject. In the present embodiment, “value determined in advance for each subject” is an example of a combined load and a bias value calculated from a sample image, which will be described later. Further, the “predetermined orientation” is a reference subject orientation set for each subject. In the present embodiment, the “predetermined orientation” is set on the system in advance or input from the operation unit 106 and is input by the user. It can be set for each subject. For example, in the case of chest front image data, the reference direction suitable for diagnosis is the orientation in which the head is anatomically above the captured image data. On the other hand, it is difficult to define in a subject whose orientation suitable for diagnosis varies depending on the user, such as extremities. Therefore, when the user sets a predetermined direction using the operation unit 106, the direction of the subject suitable for diagnosis of each user can be set.

  Furthermore, the radiation imaging apparatus 100 includes an image quality improving unit 113, a display processing unit 114, and a display unit 109. The image quality improving unit 113 performs an image quality improving process in consideration of the determined subject orientation with respect to the captured image. Based on the calculation result calculated by the calculation unit 108, the display processing unit 114 performs processing so as to match or be similar to the direction of the subject serving as a reference for setting the orientation of the subject to a predetermined direction. That is, the display processing unit 114 changes the orientation of the subject so that the orientation of the subject in the image data matches or resembles a predetermined orientation of the subject on the display unit 109 that is predetermined by the user. The display unit 109 displays the orientation of the subject in the changed image data as an image.

  The image processing apparatus 200 according to the present embodiment includes at least a calculation unit 108 and a display processing unit 114. Further, the image processing apparatus 200 may include other configurations such as the image quality improving unit 113 as necessary in the system.

  The calculation unit 108 includes a first feature amount extraction unit 110 and a determination unit 112 in order to determine the subject orientation from the captured image data output from the preprocessing unit 105. The calculation unit 108 may further include a second feature amount extraction unit 111. The first feature quantity extraction unit 110 and the second feature quantity extraction unit 111 in the present embodiment correspond to the first feature quantity extraction unit and the second feature quantity extraction unit in the present invention. In the following description, the orientation of the subject is determined using the first feature value extraction unit 110, the second feature value extraction unit 111, and the determination unit 112, but the present invention is not limited thereto. is not. In the present invention, the orientation of the subject can be determined using the first feature quantity extraction unit 110 and the determination unit 112.

  The first feature amount extraction unit 110 extracts a first feature amount based on a tendency of comparing intensity distributions of pixel values for a plurality of regions in the image data of the subject between the plurality of regions. Further, the second feature amount extraction unit 111 sets a plurality of directions in the captured image data, and based on the tendency of comparing the distribution of the change amount of the pixel value calculated for each of the plurality of directions between the plurality of directions. To extract the second feature amount. The first feature value and the second feature value will be described later. The discriminating unit 112 discriminates the orientation of the subject in the image data based on the first feature amount and a value predetermined for each subject. The determination unit 112 may determine the orientation of the subject in the image data based on the first feature amount, the second feature amount, and a value predetermined for each subject. The result of the determination of the object orientation by the determination unit 112 is defined as a direction rotated by D degrees with the reference object direction set in advance for each object as 0 degrees. As an example, FIG. 2 shows four directions when the direction in which the head is above the captured image data is defined as the reference direction for the chest front image data. In the present embodiment, the discriminating unit 112 sets D = 90 degrees, and the subject orientation discriminating process is performed in four directions of the captured image data, that is, the reference orientation for each region (0 degree), 90 degrees, 180 degrees, and 270 degrees. One of them shall be determined. In this embodiment, the direction of the subject is four directions, but is not limited to this.

  Next, using the flowchart shown in FIG. 3, the orientation of the subject is determined from the captured image data captured by the radiation imaging apparatus 100, and the image quality enhancement processing and the image rotation processing are performed in consideration of the orientation of the subject. A method of outputting to the display unit 109 will be described.

  First, in step S <b> 201, the calculation unit 108 acquires captured image data that has been pre-processed by the pre-processing unit 105 and inputs the captured image data to the first feature amount extraction unit 110 and the second feature amount extraction unit 111. . In this step, the calculation unit 108 extracts the feature amount of the subject in the captured image data.

  Next, in step S <b> 202, the calculation unit 108 inputs the feature amounts extracted by the second feature amount extraction unit 111 and the first feature amount extraction unit 110 to the determination unit 112. The determination unit 112 calculates the orientation of the subject in the captured image data based on the first feature amount, the second feature amount, and a value determined in advance for each subject.

  In step S <b> 203, the calculation unit 108 outputs the object orientation determination result output from the determination unit 112 to the image quality improvement unit 113. The image quality enhancement unit 113 performs an image quality enhancement process based on the object orientation determination result, and generates image quality processed image data. Here, the image quality enhancement process is a gradation process for converting the entire image into a gradation suitable for diagnosis using, for example, the region of interest of the subject as a reference. The image quality improving unit 113 may extract a predetermined region of interest from the subject in the image quality improving process. For example, in the case of a chest image, a lung field is extracted as a region of interest. By performing the image quality enhancement process on the image data for which the subject orientation is calculated, the image quality enhancement processing unit 113 improves the extraction accuracy of the region of interest and obtains captured image data with a higher image quality. .

  In step S <b> 204, the radiation imaging apparatus 100 inputs the object orientation determination result output from the determination unit 112 and the image quality processed image data output from the image quality improving unit 113 to the display processing unit 114. Further, the display processing unit 114 reads from the storage unit 107 the subject of the captured image data and the reference orientation corresponding to the subject. The display processing unit 114 rotates the orientation of the subject with respect to the image data that has been subjected to the high image quality processing so that the orientation of the subject in the image data matches or resembles a predetermined orientation (reference orientation for each subject). Then, the image is output to the display unit 109 for display as a display image, and the process is terminated. With the above processing, the image processing apparatus 200 can calculate the orientation of the subject in the captured image without depending on the subject, and can match the orientation specified by the user (reference orientation for each subject).

  In the following description, the horizontal direction of the captured image data is defined as the x-axis and the vertical direction is defined as the y-axis, the coordinates of the upper left corner are represented by the origin (x, y) = (1, 1), and the coordinates of the captured image data I The pixel value at (x, y) is represented by I (x, y). Further, the horizontal size of the captured image data is W and the vertical size is H. That is, the coordinates of the lower right corner of the captured image data are (x, y) = (W, H), where x is in the range of 1 to W and y is in the range of 1 to H.

(First feature)
The first feature quantity extracted by the first feature quantity extraction unit 110 is determined based on edge pixels that are extracted from the set plurality of areas by setting a plurality of areas in the captured image data of the subject. Feature amount. Various first feature quantities can be applied as long as they meet this definition. In the present embodiment, the feature quantities are calculated as follows.

  The first feature amount extraction performed by the first feature amount extraction unit 110 will be described in detail with reference to the flowchart of FIG. Steps S301 to S303 in FIG. 4A are steps in the feature amount extraction process in step S201 described in FIG. As step S301, the first feature amount extraction unit 110 calculates a change amount for each of a plurality of regions in the captured image data. In the present embodiment, the distribution of the change amount of the pixel value in the captured image data is shown as edge intensity image data Em. Here, the edge pixel is a pixel in which the change amount of the pixel value is equal to or greater than a predetermined value in the captured image data. The edge intensity image data is image data indicating the intensity distribution of edge pixels.

  First, the first feature amount extraction unit 110 calculates the edge intensity image data Em of the captured image data I using the above-described edge image data Ex and Ey in the x-axis direction and the y-axis direction according to the following equation.

  Here, as an example, when the captured image data I is chest front image data, the first feature amount extraction unit 110 calculates edge intensity image data calculated from the x-direction edge image data Ex and the y-direction edge image data Ey. Em is shown in FIG.

  Further, the first feature quantity extraction unit 110 sets the edge image data when ernum plural regions are set in the edge intensity image data Em as ERn (n = 1, 2,..., Rnum). calculate. The edge image data ERn is image data in which the pixel value in the set area is represented by the pixel value of the edge intensity image data Em and the other pixel values are represented by zero. Here, the first feature amount extraction unit 110 sets four regions of the upper half, the lower half, the left half, and the right half of the edge strength image data Em, and the edge image data ER1, ER2, ER3, and ER4 are respectively set next. Calculate based on the formula.

  Here, as an example, FIG. 6 shows region edge image data ER1 to ER4 acquired by the first feature amount extraction unit 110 from the edge strength image data Em when the captured image data I is chest front image data.

  Subsequently, in step S302, the first feature amount extraction unit 110 extracts a statistic from the set plurality of regions. As the statistic used here, an average value or variance value of pixel values in the captured image data, or the number of pixels having a pixel value equal to or greater than a threshold value when a predetermined threshold value is set is used. In the present embodiment, the first feature amount extraction unit 110 calculates average values as statistics from the four regions of the upper half, the lower half, the left half, and the right half of the edge intensity image data Em described above as eu, eb, el and er are calculated.

  In step S303, the first feature amount extraction unit 110 uses the calculated eu, eb, el, and er as a first feature amount that represents a relationship for each statistic for each of the plurality of regions. calculate. Specifically, the first feature quantity extraction unit 110 calculates eub based on the relation between eu and eb and elr based on the relation between el and er, which are expressed by the following equations, as the first feature quantity.

  The eub and elr calculated as described above are the first feature amounts that represent the distribution distribution trend (edge density) for each region set in the captured image data. For example, eub in Expression 10 is a feature value that takes a value close to 0 in the captured image data if the edge is denser in the upper half than the lower half, and close to 0 if the edge is denser in the lower half than the upper half. . Similarly, elr in Expression 11 is a feature value that takes a value close to 0 in the captured image data when the edge is denser in the left half than the right half, and close to 0 if the edge is denser in the right half than the left half. It is.

  As described above, the calculation unit 108 uses the first feature amount extracted separately from the plurality of regions as one of the features representing the subject direction of the captured image data without using the anatomical information of the subject. be able to. In the present embodiment, the imaged image data of the chest has been described, but since the trend of the distribution of changes in pixel values is different for each of a plurality of regions in the lung field, abdomen, limbs, and head, It can be used as a feature amount for calculating the orientation of the subject.

  Next, preprocessing performed by the first feature quantity extraction unit 110 before extraction of the first feature quantity will be described. The first feature quantity extraction unit 110 may perform reduced image data processing in which the captured image data of the subject is reduced to a predetermined image data quantity suitable for feature quantity extraction as preprocessing for the first feature quantity extraction. Good. For this reason, the first feature quantity extraction unit 110 can increase the speed of the feature quantity extraction process. Further, by using the reduced image data, it is possible to remove the influence of noise in the image data that is not necessary for feature amount extraction.

  The pixel values in the captured image data of the subject have various distributions depending on the radiation irradiation dose and the subject structure. Therefore, the first feature quantity extraction unit 110 may perform normalization processing of a pixel value range having various distributions between captured image data that varies depending on the radiation irradiation dose and the subject structure, as preprocessing. For example, the first feature quantity extraction unit 110 can assign a range of pixel values to a range of 0 to 1 by normalization processing. For this reason, the first feature quantity extraction unit 110 can improve the accuracy of feature quantity extraction.

  Furthermore, the first feature quantity extraction unit 110 can also perform processing based on captured image data obtained by removing a region unnecessary for determining the orientation of the subject from the captured image data of the subject. For example, in the case of radiographic image data, the first feature amount extraction unit 110 includes an area outside the irradiation field that is not irradiated with radiation, a blank area where no radiation is transmitted through the subject, and an area where metal is detected. It is possible to perform a region removal process for removing the. With the above processing, the image processing apparatus 200 can extract the first feature amount in a region from which a portion irrelevant to the feature amount of the subject is removed, so that the accuracy of calculation of the subject direction can be improved.

  The first feature quantity extraction unit 110 can switch execution or non-execution of these pre-processes depending on the feature quantity extraction processing and display processing methods, performance such as calculation accuracy and processing speed, and the target subject. Is preferable.

  In the present embodiment, the distribution tendency of the change amount of the pixel value for each of the plurality of regions is used as the first feature amount. However, a combination of two or more edge statistics of a plurality of regions may be used as the first feature amount. As a result, the accuracy of feature quantity extraction can be improved.

(Second feature)
The second feature quantity extraction performed by the second feature quantity extraction unit 111 when the second feature quantity extraction unit 111 is used will be described in detail with reference to the flowchart of FIG. Steps S401 to S403 in FIG. 4B are steps in the feature amount extraction process in step S201 described in FIG. Note that the flowchart of FIG. 4B may be processed in parallel with the flowchart of FIG. 4A, or may be processed before the flowchart of FIG.

  The second feature amount is a feature amount determined based on the distribution of edge pixels in a plurality of directions that are set in a plurality of directions in the captured image data and extracted separately from the set directions.

  By using the second feature amount for calculating the orientation of the subject, the accuracy is improved by combining it with the first feature amount even in a region where it is difficult to calculate the orientation of the subject. Specifically, the effect is large when discriminating vertically and horizontally symmetric subjects that are difficult to extract as the first feature amount.

  In the present embodiment, the second feature quantity extraction unit 111 extracts the distribution of edge pixels in the captured image data as edge image data. In the present embodiment, various second feature amounts can be applied as long as they meet this definition.

  First, in step S401, a change amount for each of a plurality of directions in the captured image data is calculated.

  First, when the captured image data is I, direction edge image data Ed obtained by extracting an edge orthogonal to a unit vector (nx, ny) indicating a certain direction can be expressed by the following equation.

  Here, in this embodiment, a plurality of directions are defined as two of an x axis (horizontal direction) and a y axis (vertical direction). The second feature amount extraction unit 111 calculates directional edge image data Ex and Ey representing edge in the x-axis direction and edge pixel in the y-axis direction from the captured image data by the following formula.

  Expressions 1 and 2 show examples when the unit vectors (nx, ny) are (0, 1) and (1, 0). Here, as an example, FIG. 7 shows x-direction edge image data Ex and y-direction edge image data Ey in the case where the captured image data I is chest front image data. The second feature quantity extraction unit 111 extracts edge pixels in the horizontal and vertical directions from the captured image data, which is grayscale image data, for the x-direction edge image data Ex and the y-direction edge image data Ey, respectively.

  As step S402, the second feature quantity extraction unit 111 may calculate a statistic for each edge image data for each of a plurality of directions in the captured image data, and extract the statistic as a second feature quantity. it can. The statistics amount extracted by the second feature amount extraction unit 111 is, for example, the edge pixel direction such as the average value or variance value of the pixel values in each edge image data, or the number of edge pixels (percentage of edge pixels). A direction edge statistic indicating a tendency for each is extracted. Here, an average value is calculated as a statistic from the above-described x-direction edge image data Ex and y-direction edge image data Ey representing the x-axis direction edge and the y-axis direction edge, and set as ex and ey.

  Next, as step S403, the second feature quantity extraction unit 111 uses the calculated direction edge statistics ex and ey as a second feature quantity that represents the relationship between each of the plurality of direction edge statistics. calculate. Then, the second feature amount extraction unit 111 calculates ed specifically represented by the following equation as the second feature amount.

  Note that ed calculated above is a feature amount representing a dominant direction among a plurality of direction edge statistics extracted from the captured image data. For example, Expression 17 is 1 if the direction edge statistic in the y-axis direction is more dominant than the direction edge statistic in the x-axis direction in the captured image data, and the direction edge statistic in the x-axis direction is 1 in the y-axis direction. If it is more dominant than the direction edge statistics, it takes a value close to 0. Since the amount of the edge of each part of the human body that is the subject differs depending on the direction, the feature amount can be extracted regardless of the subject. In the present embodiment, the imaged image data of the chest has been described. However, in the lung field, abdomen, limbs, and head, the amount of edge varies depending on the direction, so that the orientation of the subject is calculated. It can be used as a feature amount.

  As described above, the second feature amount determined based on the change amount of the pixel value in the plurality of directions separately extracted from the plurality of directions by the second feature amount extraction unit 111 is the anatomical structure of the subject. Can be used to calculate the orientation of the captured image data without using.

  In the present embodiment, the second feature quantity extraction unit 111 extracts the second feature quantity as described above, but the present invention is not limited to this. For example, ex and ey may not be average values but may be the ratio of pixels having a pixel value equal to or greater than a predetermined threshold T as in the following equation, and ed may be calculated based on these values. Furthermore, a plurality of directional edge statistics can be calculated for each edge image data in a plurality of directions, and can be used as one of the second feature values according to the relationship for each calculated statistic.

  Here, the second feature quantity extraction unit 111 performs division by the number of pixels H × W of the entire photographed image data in order to reduce the influence of the photographed image data having different image data sizes in the above equation.

  In this embodiment, the direction edge relation amount is used as the second feature amount. However, as the second feature amount, a combination of two or more of the plurality of statistics described above may be used.

  In this embodiment, the plurality of directions are the vertical direction and the horizontal direction. However, the direction in which the second feature amount is extracted may be further increased and used as a statistic. The computing unit 108 can improve the discrimination accuracy of the subject that is difficult to discriminate by the discriminating means described later by increasing the direction in which the second feature amount is extracted.

  Further, the second feature quantity extraction unit 111 can perform the same preprocessing as that performed by the first feature quantity extraction unit 110 described above on the captured image data. Accordingly, the second feature quantity extraction unit 111 can extract the second feature quantity from the pre-processed captured image data, and the same effect can be obtained.

(Subject orientation discrimination processing)
Various methods can be applied to the discriminating unit 112 that outputs a subject orientation discrimination result using a feature amount as an input. In the present embodiment, the discriminating unit 112 discriminates the direction of the subject using a three-layer feedforward neural network (hereinafter referred to as FFNN) including an input layer, an intermediate layer, and an output layer, and outputs a discrimination result. By using FFNN, it is possible to relate the relationship between the above-described feature amount and the direction of the subject by setting a value (binding load, bias value) determined for each subject.

  The three-layer FFNN of the discriminating unit 112 is composed of an input layer composed of Nin neurons, an intermediate layer composed of Nmid neurons given by parameters, and Nout neurons corresponding to the number of classes to be discriminated. Output layer. The neurons of the input layer and the intermediate layer, and the intermediate layer and the output layer are connected by connection weights, and a bias value is added to the intermediate layer and the output layer. This joint weight and bias value are the discrimination parameters of the three-layer FFNN. The determination unit 112 can classify the feature amount given to the input layer according to the value indicated by the output layer by reading and setting an appropriately set value from the storage unit 107.

Nin neurons belonging to the input layer are represented by a Nin-dimensional vector x = (x ₁ , x ₂ ,..., X _Nin ) indicating the value of the i-th neuron as x _i . This vector x corresponds to a feature amount used for discrimination. Accordingly, in the subject discrimination process, a vector having the first feature value and the second feature value extracted in step S201 as elements is created and used as the value of the vector x. In the present embodiment, three of eub and elr are used as the first feature quantity, and ed is used as the second feature quantity, and Nin = 3, x ₁ = ed, x ₂ = eub, and x ₃ = elr 3 The dimension vector x becomes a neuron of the input layer.

Nmid neurons belonging to the intermediate layer are represented by an Nmid dimension vector a = (a ₁ , a ₂ ,..., A _Nmid ) indicating the value of the i-th neuron as a _i . This vector a is calculated from the feature value given to the input layer. Here, the j-th element a _j of the vector a is expressed by the following equation using the Nin-dimensional vector x representing the neurons of the input layer.

Further, Nout neurons belonging to the output layer are represented by Nout dimensional vector o = (o ₁ , o ₂ ,..., O _Nout ). This vector o is calculated from the value obtained in the intermediate layer. Here, the j-th element o _j of the vector o is expressed by the following equation using the Nmid-dimensional vector a representing the intermediate layer neuron.

The number Nout of classes to be identified here corresponds to the number of orientations of the object to be identified here, and each element of the Nout dimension vector o is an evaluation value indicating which object direction the captured image data is. In this embodiment, Nout = 4 discrimination directions are set, and o ₁ , o ₂ , o ₃ , and o ₄ are set as evaluation values corresponding to the subject orientations of 0 degree, 90 degrees, 180 degrees, and 270 degrees, respectively. To do. When the desired orientation of the subject is determined, only one element of the vector o takes a value close to 1, and the other elements take values close to zero. Accordingly, in the three-layer FFNN, the determination unit 112 outputs the object direction corresponding to the element having the maximum value among the elements of the vector o as the object direction determination result, thereby achieving the object direction determination process. .

In Expression 20, w ^{1} _ji is a connection load connecting the neuron i in the input layer and the neuron j in the intermediate layer, and b ^{1} _j is a bias value applied to the neuron j in the intermediate layer. In Expression 21, w ^{2} _ji is a connection load connecting the neuron i in the intermediate layer and the neuron j in the output layer, and b ^{2} _j is a bias value applied to the neuron j in the output layer. These are discrimination parameters determined for each subject that have been adjusted to appropriate values in advance by machine learning described later.

The determination unit 112 needs to set the determination parameter to an appropriate value in advance in order to perform correct determination in the object determination process. In the above-described three-layer FFNN, the coupling weights w ^{1} , w ^{2} , bias values b ^{1} , b ^{2} are discrimination parameters for generating an appropriate discrimination result, Adjusted to the value. In this embodiment, supervised learning which is one of machine learning is used.

  In supervised learning, a large number of sample image data for each subject and a correct discrimination result of each sample image data are prepared as teacher data, and the result of giving the sample image data to the discrimination unit 112 matches the teacher data. Learning to adjust the discrimination parameter. Supervised learning is performed for each subject, and a discrimination parameter for each subject is generated and stored in the storage unit 107 in advance.

In the three-layer FFNN, the teacher data is the same Nout-dimensional vector as that of the output layer, and the value of the element corresponding to the expected subject direction is 1 and the value of the other elements is 0. Then, a feature amount is generated from the sample image data and given to the input layer of the three-layer FFNN, and the discrimination parameters w ^{1} , w ^{2} , b ^{ are set so that the generated output layer value is equal to the teacher data. ^1} and b ^{2} are adjusted. This adjustment may be performed as long as the discrimination parameter is accurately determined. For example, an error back propagation method is used.

  In the above description, the case in which the object orientation determination process is performed using the determination parameter generated by supervised machine learning using the three-layer FFNN is described, but other determination means may be used.

  For example, if the number of intermediate layers of the three-layer FFNN described in the present embodiment is increased, it is possible to learn how to combine a plurality of feature amounts in the machine learning process. In this case, for example, the object orientation determination process can be performed even if the direction edge image data itself is used as the second feature amount.

(About information processing equipment)
In addition, the present invention supplies a software program (in the embodiment, a program corresponding to the flowchart shown in the drawing) to the system or apparatus directly or remotely, and the system or apparatus implements the functions of the above-described embodiments. The computer can read out and execute the supplied program code.

  Accordingly, since the functions of the present invention are implemented by computer, the program code installed in the computer also implements the present invention. In other words, the present invention includes a computer program itself for realizing the functional processing of the present invention.

  As mentioned above, although preferable embodiment of this invention was described, it cannot be overemphasized that this invention is not limited to these embodiment, A various deformation | transformation and change are possible within the range of the summary.

108 Calculation unit (corresponding to calculation means)
109 Display (corresponding to display means)
114 Display processing unit (corresponding to display processing means)
200 Image processing apparatus

Claims (20)

In order to associate a value indicating the tendency of comparing the intensity distribution of pixel values for each of a plurality of regions in the image data of the subject between the plurality of regions, the value indicating the tendency and the orientation of the subject in the image data in advance. A calculation means for performing a calculation for determining a direction of the subject in the image data based on a value determined for each sample;
Based on the calculation result calculated by the calculation means, the subject in the subject image is in a predetermined orientation with respect to the display means for displaying the subject image based on the image data. Display processing means for processing to display an image of the subject on the display means;
An image processing apparatus comprising:   The display processing means determines the orientation of the subject in the image data so that the orientation of the subject in the image data matches or is similar to the orientation of the subject serving as a reference for setting the predetermined orientation. The image processing apparatus according to claim 1, wherein the image processing apparatus is changed and displayed on the display unit. The computing means is
A first feature amount extraction unit configured to extract a first feature amount based on a tendency of comparing intensity distributions of pixel values calculated for each of the plurality of regions between the plurality of regions; The image processing apparatus according to claim 1, wherein the orientation of the subject in the image data is determined based on the amount. The first feature amount extraction unit extracts a statistic for each of the plurality of regions,
The image processing apparatus according to claim 3, wherein the first feature amount includes a value corresponding to a statistical amount relationship for each of the plurality of regions. The first feature amount extraction unit extracts a statistic for each of the plurality of regions,
The image processing apparatus according to claim 3, wherein the first feature amount includes the statistical amount.   The first feature amount is obtained by extracting an edge pixel whose pixel value change amount in the image data of the subject is equal to or larger than a predetermined value, and an upper half region and a lower half region or a left half portion and a right half in the image data. 6. The value according to claim 3, wherein the value indicates a tendency of an edge pixel distribution among the plurality of regions extracted based on a result of comparing statistics of at least one set of the regions. The image processing apparatus according to any one of the above. The computing means is
The image processing apparatus further includes second feature amount extraction means for extracting a second feature amount based on a tendency of comparing distributions of change amounts of pixel values calculated for a plurality of directions in the captured image between a plurality of directions. ,
The image processing apparatus according to claim 1, wherein an orientation of the subject in the image data is determined further based on the second feature amount. The second feature amount extraction means includes
Calculating a statistic for each of the plurality of directions from a distribution of edge pixels in which the change amount of the pixel value is equal to or greater than a predetermined value, and setting the relationship between the statistics for the plurality of directions as a second feature quantity The image processing apparatus according to claim 7, wherein the apparatus is an image processing apparatus. The second feature amount extraction means includes
8. The method according to claim 7, further comprising: calculating a statistic for each of the plurality of directions from a distribution of edge pixels having a change amount of the pixel value equal to or greater than a predetermined value, and setting the statistic as a second feature quantity. 8. The image processing apparatus according to 8.   The second feature amount is an edge pixel in which a change amount of a pixel value in at least two directions of a change amount of a pixel value in a horizontal direction and a change amount of a pixel value in a vertical direction in the image data of the subject is a predetermined value or more. 10. The value according to claim 7, wherein the value indicates a tendency of the distribution of edge pixels extracted based on a result of comparing the statistics of the edge pixels in the two directions. The image processing apparatus described.   7. The statistic includes at least one of an average value of pixel values, a variance value of pixel values, and a number of the edge pixels in the image data. The image processing apparatus according to any one of 8 to 10. The computing means is
The image processing apparatus according to claim 1, wherein the orientation of the subject is calculated based on a value predetermined for each subject by machine learning. The computing means is
The orientation of the subject serving as a reference for setting the predetermined orientation is associated with the value predetermined for each subject, and the value indicating the tendency and the value predetermined for each subject The image processing apparatus according to claim 1, further comprising: a determination unit configured to determine a direction of the subject in the image data based on A storage unit is further provided for storing a value determined for each subject and information on the subject including a value predetermined for each subject that is associated with the reference orientation of the subject. And
The calculation means performs a calculation for determining the orientation of the subject in the image data based on a value indicating the tendency and a value predetermined for each subject acquired from the storage unit. The image processing apparatus according to claim 1.   The image processing according to any one of claims 1 to 14, wherein the calculation unit calculates the orientation of the subject based on reduced image data obtained by reducing the image data to a predetermined data amount. apparatus.   The display processing means determines the orientation of the subject in the image data so that the orientation of the subject in the image data matches or is similar to the orientation of the subject serving as a reference for setting the predetermined orientation. The image processing apparatus according to claim 1, wherein the image processing apparatus is rotated and displayed on the display unit. Feature amount extraction means for extracting a value indicated by a distribution of change amounts of pixel values for a plurality of regions in the image data of the subject;
A discriminating means for discriminating a direction of the subject in the image data based on a value indicated by a tendency of the distribution of the change amount of the pixel value;
An image processing apparatus comprising: The image processing apparatus according to claim 1, wherein the image data of the subject is image data based on a captured image of the subject irradiated with radiation. A value indicating a tendency of distribution of pixel value variation for each of a plurality of regions in the image data of the subject, a value predetermined for each subject in order to associate the value indicating the tendency and the orientation of the subject, A calculation step of performing a calculation for determining the orientation of the subject in the image data based on
Based on the calculation result calculated by the calculation step, the image of the subject is displayed on the display means so as to be in a predetermined orientation with respect to the display means for displaying the image of the subject based on the image data. A display processing step for performing processing;
An image processing method comprising:   A program for causing a computer to execute the image processing method according to claim 19.

Images