JP2021022028A - Image processing system, image processing method and image processing program - Google Patents

Abstract

To provide an image processing system, an image processing method, and an image processing program that identify an area of an object (for example, a reinforcing bar) with high accuracy from an image photographed by a camera.SOLUTION: An image processing system 10 of present invention includes: area of interest determination means 13 that calculates, for a processing object image 22 extracted from an image, a representative value 24 of pixels in each column direction and determines, based on a feature point 26 in a row direction of the representative value 24, an area of interest 30 for the object OB; threshold determination means 17 that determines thresholds for each area of interest 30; and generation means 19 that generates, based on the threshold value, a binary image 35 of the area of the object OB and an area excluding the object OB. The area of the object OB is identified from the photographed image of the object OB.SELECTED DRAWING: Figure 4

Description

The present invention relates to an image processing apparatus, an image processing method, and an image processing program for identifying an area of the object from a captured image of the object.

Conventionally, in various fields, a method of identifying an area of an object from an image of the object has been adopted.

As an example, at construction sites, bar arrangement inspections are conducted to prevent construction mistakes and inspection mistakes related to the reinforcing bars of reinforced concrete buildings. In the bar arrangement inspection, it is necessary to grasp the number, diameter, pitch, etc. of the reinforcing bars used, but since it takes a lot of time and effort to record all the parts, image analysis processing is used as a means to support the bar arrangement inspection. Has developed a reinforcing bar area extraction device that automatically measures the number of reinforcing bars, diameter, and the like. In the conventional reinforcing bar area extraction device, for example, a background bar (white sheet) is installed on the background of the reinforcing bar to make it easier to see, the image is taken with a camera, and the boundary (edge) between the reinforcing bar and the background is specified by image analysis processing. By doing so, the region of the reinforcing bar is extracted (see, for example, Patent Documents 1 and 2).

Japanese Patent No. 5841760 Japanese Patent No. 5412092

However, the technique described in Patent Document 1 and the like may not be able to accurately extract the reinforcing bar due to a problem at the time of taking an image which is the basis of image analysis processing.

Specifically, a white background bar is placed on the background of the reinforcing bar to clarify the area of the reinforcing bar, but when shooting the reinforcing bar with a camera (for example, a compact digital camera), sunlight or a flash of the camera When the reinforcing bar is irradiated with, overexposure occurs in a part of the reinforcing bar, or the image shows the shadow of the reinforcing bar. In that case, there is a problem that the reinforcing bar cannot be extracted accurately, such as recognizing the area of the reinforcing bar as the background or recognizing the shadow part as the reinforcing bar in the image analysis process.

The present invention has been made in view of such a problem, and provides an image processing device, an image processing method, and an image processing program for identifying an area of an object (for example, a reinforcing bar) with high accuracy from an image taken by a camera. is there.

The present invention is an image processing device that identifies a region of an object from an image of the object, and is a first array direction extracted from the image and a second array direction orthogonal to the first array direction. For the image to be processed containing the plurality of pixels of, the representative value of the pixel is calculated for each of the first array directions, and the region of interest for the object is determined based on the feature points of the representative value in the second array direction. A region of interest determining means, a threshold determining means for determining a threshold for each region of interest, and a generating means for generating a binary image of a region of the object and a region excluding the object based on the threshold. It is related to an image processing apparatus characterized by having.

Further, the present invention is an image processing method for identifying a region of an object from an image obtained by photographing the object, and is a second arrangement direction extracted from the image and a second array orthogonal to the first arrangement direction. For a processing target image including a plurality of pixels in the arrangement direction, a representative value of pixels for each of the first arrangement directions is calculated, and based on the feature points of the representative value in the second arrangement direction, interest in the object. Generation of generating a binary image of the region of the object and the region excluding the object based on the threshold, the step of determining the region of interest that determines the region, the step of determining the threshold for each region of interest. It relates to an image processing method characterized by having steps.

The present invention also relates to an image processing program for causing a computer to execute the above image processing method.

According to the present invention, it is possible to provide an image processing device, an image processing method, and an image processing program that identify a region of an object (for example, a reinforcing bar) with high accuracy from an image taken by a camera.

It is a figure which shows the outline of the image processing apparatus which concerns on this embodiment, (A) is a block diagram which shows an outline example of a hardware configuration, (B) is a block diagram which shows an outline example of a functional configuration. .. (A) It is a figure which shows the photographed image which concerns on this Embodiment, and (B) is a figure which shows the image to be processed. It is a schematic diagram which shows the method of determining the region of interest, (A) is the figure which shows an example of the image to be processed, and (B) is the graph which shows the change (transition) of the representative value in the row direction. It is a schematic diagram which shows the method of determining the region of interest, (A) is the figure which shows by extracting the vicinity of one object of the processing target image, (B) is the schematic diagram which shows the plurality of pixel groups of (A). Yes, (C) is a diagram showing the vicinity of one object of the processing target image extracted, and (D) is a diagram showing an example of a change in the row direction of the representative value corresponding to (C). (A) is a diagram showing an example of the image to be processed after the region of interest is determined, and (B) is a histogram showing the luminance distribution of the pixels included in the region of interest 30. (A) is an example of a photographed image, (B) is a diagram showing an example of an area of interest 30, and (C) is an example of a binary image. It is a flow figure which shows an example of the processing flow of the image processing method which concerns on this embodiment. It is a flow figure which shows an example of the flow of interest area determination processing of this embodiment. It is a flow figure which shows an example of the flow of the threshold value determination process of this embodiment. It is a flow figure which shows an example of the flow of the generation process of this embodiment.

<Image processing device>
Hereinafter, the best mode for carrying out the present invention will be described with reference to the drawings. The image processing device 10 according to the embodiment of the present invention identifies a region of the object from an image obtained by photographing the object. The image processing device 10 of the present embodiment can be used in various fields, and the object of the present embodiment is, for example, an object that can be photographed as a still image by a camera (for example, a compact digital camera). As an example, the image processing device 10 is preferably applied to a part of means for supporting a bar arrangement inspection at a construction (construction) site or the like, and a case where the object is a reinforcing bar will be specifically described below. ..

1A and 1B are diagrams showing an outline of the image processing apparatus 10 of the present embodiment, FIG. 1A is a block diagram showing an outline of a hardware configuration, and FIG. 1B is an outline of a functional configuration. It is a block diagram which shows an example.

With reference to FIG. 1A, the image processing apparatus 10 is an apparatus capable of carrying out the image processing method according to the present embodiment. The image processing device 10 is, for example, a computer having a control means 1, a storage means 3, an input means (input interface) 5, an output means (output interface) 7, a communication means 9, and the like. More specifically, the image processing device 10 includes, for example, a stand-alone personal computer (PC), a client terminal device (PC) that connects to a server device via a network, a mobile communication network, or a nearby device. It may be a smartphone or tablet that can be connected to a distance communication network, or a mobile terminal such as a PDA (personal digital assistant (personal data assistant)).

The control means 1 includes an arithmetic processing unit (CPU; central processing unit, MPU; microprocessor unit, etc.) and performs various arithmetic processes and various control processes for hardware. Further, the control means 1 executes an operating system (OS; Operating System) stored (installed) in the storage means 3, various application programs, and browser software. Further, the control means 1 executes the image processing of the present embodiment by reading and executing the image processing program of the present embodiment stored in the storage means 3. That is, a part of the control means 1 functions as an image processing means.

The storage means 3 is a main storage device (main memory (RAM; Random Access Memory), storage, ROM (Read Only Memory), etc.), but may include an external storage device. The storage means 3 includes OS, firmware, browser software, various application programs, various drivers, and various data (image (imaging) data obtained by photographing an object, image data obtained by processing the image data, and other image data. Data that can be processed by various application programs such as text data, documents) can be stored and saved.

The input means 5 is, for example, a means for receiving a command or the like from a user who operates the image processing device 10, and includes an operation means such as a keyboard, a mouse, and a touch panel. Further, the input means (input interface) 5 is, for example, an image (imaging) data obtained by photographing an object or image data obtained by processing the image data (hereinafter, not shown) via a communication means 9 or an external input means (not shown). Image data simply accepts input (also referred to as "image").

The output means (output interface) 7 is a means for outputting an image processing result or the like, and is, for example, an interface for outputting to a display means such as a display and / or a printer.

The communication means 9 is a means for communicating with an external device via a communication line (for example, the Internet or a mobile communication network), but the communication means 9 may not be provided.

The function of the image processing apparatus 10 according to the present embodiment will be described with reference to FIG. The image processing device 10 includes an image acquisition means 11, a region of interest determination means 13, a correction means 15, a threshold value determination means 17, a generation means 19, and the like, and is obtained from an image (captured image) obtained by photographing an object. Identify the area of the object.

The image acquisition means 11 acquires a processing target image including an image of the object. The image to be processed is an image that is input data for image processing performed by the image processing device 10 of the present embodiment. More specifically, it is an image obtained by converting an image (photographed image) of an object (for example, a reinforcing bar) into a grayscale image and extracting (cutting out) a predetermined range including a plurality of objects to be analyzed, for example. ..

FIG. 2 is a diagram showing an example of a captured image 21 and a processing target image 22. FIG. 2A is a captured image 21, and FIG. 2B is a processing target image 22.

As shown in FIG. 2A, the photographed image 21 is photographed by a camera, for example, at a construction (construction) site or the like, in which a bar-shaped sheet (background bar) BS is installed in the background of a reinforcing bar which is an object OB. It is an image that was made. The background bar BS is a sheet for making the object OB easier to see, and is preferably a single color having a color tone difference and a contrast difference (contrast difference) with the object OB, and is a white sheet here.

As shown in FIG. 2B, the processed image 22 includes a background BG (a part of the background bar BS) and at least one object (reinforcing bar) OB after converting the captured image 21 into a grayscale image. In addition, it is an image obtained by extracting (cutting out) a part (indicated by a thick line in the figure (A)) in a rectangular shape. Further, the image to be processed 22 includes a plurality of pixels in the first array direction and the second array direction orthogonal to the first array direction. Here, the first arrangement direction is, for example, the column direction (the direction of the same column, the vertical direction of the paper surface, the vertical direction), and the second arrangement direction is the row direction (the direction of the same row, the horizontal direction of the paper surface, the horizontal direction). That is, for example, the image to be processed 22 has n (n ≧ 2) pixels in the column direction and m (m ≧ 2) pixels in the row direction (horizontal and horizontal directions) (pixels in n rows and m columns). Is a collection of images. Further, the reinforcing bar which is the object OB has at least one axis thereof along the first arrangement direction (vertical direction) and is included in the processing target image 22. In this example, the image to be processed 22 includes an image of five reinforcing bars arranged so that the axes are along the vertical direction (the axes are along the first arrangement direction) as the object OB, and is an actual object. The case where the shadow SD is reflected adjacent to the object OB and there is a region having brightness between the object OB and the background BG is illustrated. In the case of the photographed image 21 taken outdoors or indoors under an irregular light source, the shadow SD is often generated adjacent to the object OB as shown in FIG. (B), and in that case, the photographed image 21 Is converted into a grayscale image, and as shown in FIG. 3B, an unintended region having a brightness value lower than the white color of the background BG exists around the object OB.

In the present embodiment, even in such a case, the shadow SD can be eliminated as much as possible to specify the position of the object OB. Therefore, the example shown in FIG. 3B is used, but the shadow SD does not occur. Of course, it can be carried out in the same manner even in the case, and the same effect can be obtained.

Here, the image acquisition means 11 only needs to be able to acquire the processing target image 22 (grayscale image in which the object OB can be easily identified) as shown in the figure (B), and the captured image 21 is the photographed image (A). It is not limited to the image shown in. Further, the image acquisition means 11 may acquire a captured image 21 (stored in the storage means 3) received from an external device or the like via the input means 5, and extract (process) the image to be processed 22. Then, the processed target image 22 (stored in the storage means 3 and processed) directly received from an external device or the like via the input means 5 may be acquired. Further, the number of objects (reinforcing bars) included in the processing target image 22 may be singular rather than plural. Further, the processing target image 22 is not limited to the gray scale image, and may be an image expressed by one or a plurality of gradations of R, G, and B, but the gray scale image can specify the target object. Easy and suitable.

With reference to FIG. 1B again, the region of interest determination means 13 sets a representative value of pixels for each first arrangement direction (for example, each column direction) of the processing target image 22 acquired by the image acquisition means 11. calculate. Then, the region of interest (ROI) 30 for the object is determined based on the feature points in the second arrangement direction (for example, the row direction) of the representative values.

3 and 4 are schematic views illustrating a method for determining the region of interest 30. FIG. 3A is a diagram showing an example of the image 22 to be processed, and FIG. 3B is a graph showing a change (transition) of a representative value in the row direction. Further, FIG. 4 is an enlarged view showing an OB portion of one object in FIG. 3, and FIG. 4A and FIG. 4C are one object of the processing target image 22 shown in FIG. 3A. It is a figure which shows by extracting the vicinity of object OB, FIG. (B) is a schematic view which shows a plurality of pixel groups 23 of FIG. (A), and FIG. (D) is an object of FIG. It is a figure which shows an example of the change in the row direction (horizontal direction, horizontal direction) of OB and the corresponding representative value (average luminance value) 24.
Note that FIG. 4B shows one pixel enlarged for convenience of explanation.

First, the region of interest determination means 13 calculates a representative value in each column direction for the image 22 to be processed. Specifically, first, as shown in FIGS. 3 (A), 4 (A), and (B), the leftmost first column of the processing target image 22, which is a set of pixels in n rows and m columns. A plurality of pixel groups 23 (23A, 23B ... 23X ... 23m) are sequentially cut out in each column direction (vertical direction, vertical direction). Each of the cut out pixel groups 23 includes n pixels Px in the column direction as shown in FIG. 4 (B).

The "representative value in each column direction" is a representative value for each pixel group 23 (23A, 23B ... 23X ... 23m) extracted for each column, and for example, n elements constituting each pixel group 23. It is a value obtained by averaging the brightness of the pixels of (the average brightness value of the pixels Px (pixel group 23) in each column).

The region of interest 13 determines representative values (average luminance value 24 (24A, 24A,) for each of the pixel groups 23 from one end to the other end (from the first row to the mth row) of the image 22 to be processed in the horizontal (horizontal) direction. 24B ... 24X ... 24m))) is calculated, and graphs (FIGS. 3B and 4D) showing changes in the obtained average luminance value 24 in the row direction (horizontal direction, horizontal direction) are acquired. FIG. 3 (B) is shown in FIG. 3 (A), and FIG. 4 (D) is shown in correspondence with the processing target image 22 (object OB) corresponding to FIG. 3 (C).

With reference to FIGS. 4 (A) and 4 (B), for example, the pixel group 23A is a pixel group of the background BG portion, and all n pixels have the maximum luminance (white), so that the average luminance value is obtained. 24A has the maximum brightness (white). Further, the pixel group 23X is a pixel group 23 near the center of the object (reinforcing bar) OB, all of the n pixels have relatively low brightness, and the average brightness value 24X is a value close to the minimum brightness (black). It becomes. Further, the pixel group 23B is a pixel group near the boundary between the background BG portion and the object OB, and the n pixels are a mixture of high-luminance and low-luminance, and the average brightness value 24X is an intermediate brightness. It becomes (gray).

In this way, the representative value (average luminance value) 24 is calculated for all the columns of the image to be processed 22 and plotted to obtain a change in the representative value 24 shown in FIG. This change in the representative value indicates the tendency of the brightness in the row direction (horizontal direction).

Next, the region of interest determination means 13 determines the feature point 26 at the representative value 24. The feature point 26 is a representative value 24 which is a feature of allocating whether the tendency of the luminance is large or small (white / black), and specifically, the boundary between the magnitude of the luminance (hereinafter referred to as “luminance distribution threshold 25””. It is a representative value 24 having the same value as the brightness of (.).

Here, as an example, of all the representative values (average brightness values) 24 shown in FIG. 3 (B), the maximum average brightness value (maximum average brightness value) 24MAX and the minimum average brightness value (minimum average brightness value) The brightness value (average value) in the middle of 24MIN is set as the brightness distribution threshold 25. In FIG. 4D, for convenience of explanation, the maximum average luminance value 24MAX and the minimum average luminance value 24MIN are shown in one object OB, but the processing is actually performed as shown in FIG. 3B. The maximum average luminance value 24MAX and the minimum average luminance value 24MIN in the target image 22.

As shown in FIGS. 3 (B) and 4 (D), the region of interest 13 has a feature point 26 at the intersection of the luminance distribution threshold value 25 and the representative value 24. In the case of an image in which background BGs are arranged on both sides of the object OB as in the present embodiment, theoretically, the area where the brightness is high changes to the area where the brightness is low after passing through the feature point 26 (26A). It passes through the feature point 26 (26B) again and becomes a region having high brightness. That is, when the image 22 to be processed is scanned (traced) from the left end, for example, the area between the first feature point 26A and the next feature point 26B is a region where the brightness is low and the presence of the target object OB is estimated. .. In practice, in addition to the two feature points 26A and 26B, an unintended feature point 26 may be included, which will be described later.

The region of interest 13 determines a mask 27 that sequentially sets two adjacent feature points 26, for example, from the left end of the image 22 to be processed, as a set and divides a section having a tendency of high brightness and a section having a tendency to have low brightness. That is, as shown in FIG. 3B, one mask 27 (27A) using, for example, the first feature point 26A and the second feature point 26B from the left along the row direction (horizontal direction, horizontal direction). ) Is set, and one mask 27B is set using the third feature point 26C and the fourth feature point 26D.

As shown by the alternate long and short dash line in FIG. 4D, the mask 27 is a rectangular region having a long side of a vertical line segment including a set of feature points 26, and theoretically corresponds to an object OB. ing. That is, as shown in FIG. 3B, masks 27A, 27B, 27C ... Are set corresponding to the objects OB1, OB2, OB3 ..., respectively.

Hereinafter, for convenience of explanation, in the graph of changes in the representative values 24 arranged in the row direction, a section having a luminance distribution threshold value of 25 or less (a section having a tendency to have low luminance) is referred to as an “ON section” and a section exceeding the luminance distribution threshold value 25 (a section having a tendency to have low luminance) The section where the brightness tends to be high) is called an "OFF section". That is, the mask 27 can be said to be a mask that divides the ON section and the OFF section.

Here, referring to FIG. 3, the inside (near the center) of the object (reinforcing bar) OB is continuously turned on. However, for example, especially in the case of bar arrangement inspection at a construction site, the reinforcing bar OB, which is the object, is in a fixed state outdoors or indoors in irregular ambient light, and is in an optimum state for image processing. It is often difficult to shoot.

Specifically, when the reinforcing bar OB is irradiated with strong sunlight or a camera flash, the surface of the reinforcing bar OB becomes bright (shining), and as a result, an OFF section is formed inside the object OB. It may occur. In this case, although the image should be processed as the object (reinforcing bar) OB, there is a problem that the reinforcing bar OB is erroneously recognized as thinner than the actual object due to the occurrence of the unintended feature point 26 and the OFF section.

Therefore, the region of interest determination means 13 is a correction means (means for performing overexposure correction processing) for preventing an unintended OFF section from occurring, that is, eliminating an unintended feature point 26 when it exists. It has 15 (see FIG. 1 (B)). Thereby, in the present embodiment, it is possible to recognize the region of the object OB as a continuous ON section even if the image is taken in irregular ambient light outdoors or indoors (object OB). One mask 27 can be set for each), and the accuracy of identifying the area of the object OB is improved. The details of the correction process of the correction means 15 will be described later.

Then, the area of interest determination means 13 determines a set of areas of interest 30 arranged in the row direction for one object OB based on the mask 27 set for each object OB. Explaining in detail with reference to FIG. 4D, first, the width L1 in the row direction (horizontal direction) and the width L2 of 1/2, for example, are calculated for one mask 27. After that, a rectangular area having a width L2 of 1/2 of the mask 27 secured in the row direction (horizontal direction) with the two feature points 26 (26A and 26B) as centers is specified in the image 22 to be processed. Is determined as the region of interest 30. That is, as shown in FIG. 4C, a pair of left and right interest regions 30 (30A, 30B) are determined for one object OB, and as shown in FIG. 3A, the processing target is determined. A set of regions of interest 30 is determined for all object OBs in image 22.

For each of the regions of interest 30 determined in this way, the region of the object OB is analyzed by the threshold value determining means 17, and the target OB and the other regions are binarized by the generating means 19.

The threshold value determining means 17 will be described with reference to FIG. FIG. (A) is a diagram showing an example of the processing target image 22 after the region of interest 30 is determined, and FIG. (B) shows the brightness distribution of pixels included in a certain region of interest 30 (30A). This is an example of a histogram (smoothed), where the horizontal axis is the luminance value [nt] and the vertical axis is the number of pixels [pieces].

The threshold value determining means 17 calculates a luminance histogram (also simply referred to as “histogram” in the following description) for all the pixels in the interest region 30 for each interest region 30, and performs smoothing processing. The smoothing process will be described later. The histogram obtained in this way is shown in FIG.

Based on this histogram, the threshold value determining means 17 determines a threshold value (hereinafter, referred to as “binarization threshold value”) for expressing the object OB and a portion other than the object OB in a binarized image. Here, the shadow SD of the object OB is also included in the portion other than the object OB. Explaining the figure (B) as an example, the threshold value determining means 17 estimates that the first peak from the minimum luminance value side is the object OB (reinforcing bar) in the histogram for each region of interest 30, and the second The mountain is estimated to be the shadow of the object OB, and the third mountain is estimated to be the background BG. In this way, even when the object OB and its shadow SD exist in the captured image 21 and the boundary between the object OB and its shadow SD is not clear in the processing target image 22 (grayscale image), FIG. By creating a histogram of the region of interest 30 as shown in (B), the object OB and its shadow SD can be clearly separated, that is, the edge portion of the object OB can be specified.

Then, the threshold value determining means 17 sets the CNT at the midpoint between the peak PK1 of the first mountain (object OB) and the peak PK2 of the second mountain (shadow), in detail, the binarization threshold value. To decide. In this example, the binarization threshold of the region of interest 30 (30A) is 130 nt. The binarization threshold is determined for the other region of interest 30B and all other regions of interest 30.

Next, the generation means 19 will be described with reference to FIG. FIG. (A) is an example of the original captured image 21, FIG. (B) is a diagram showing an example of the determined region of interest 30, and FIG. (C) is generated by the generation means 19. This is an example of the binary image 35.

The generation means 19 generates a binary image of the region of the object OB and the region excluding the object OB based on the binarization threshold value. Specifically, first, each pixel in each region of interest 30 is binarized. That is, a pixel having a brightness value smaller than the binarization threshold value is defined as a brightness value indicating white, and a pixel having a brightness value larger than the binarization threshold value is defined as a brightness value indicating black.

Then, for example, two sets of interest regions 30 are specified from the left end of the processing target image 22, and the region sandwiched between the pair of interest regions 30 is estimated to be the object OB and binarized. Specifically, the pixels in the ON section have a luminance value indicating white, and the pixels in the OFF section have a luminance value indicating black.

In the case of FIG. 3B, the pairs 301, 302 ... Of the regions of interest are specified by the two regions of interest 30 in order from the left, and the regions C1, C2 ... Of the regions of interest 30 sandwiched between the groups 301, 302 ... Is set to the same pixel value as the binarized pixel value of the adjacent region of interest 30. Further, the regions G1, G2 ... Between the sets 301, 302 ... Are presumed to be the background and have the same pixel value as the other binarized pixel value of the adjacent region of interest 30. That is, it is painted in the same color as the edge portion of the object OB.

For details, referring to the leftmost set 301 of the region of interest in FIG. (B), the pixel value W (for example, “1”) indicating white for the region of interest 30A on the left side is shown in FIG. A binary image consisting of a pixel value B (for example, “0”) indicating black and black is generated, and for the region of interest 30B on the right side, a pixel value W indicating white and a pixel indicating black are shown in FIG. Generate a binary image consisting of the value B. Then, the region C1 sandwiched between the regions of interest 30A and 30B shows the same white as the binarized pixel value W of the adjacent region of interest 30A and the binarized pixel value W of the adjacent regions of interest 30B. Let the pixel value be W.

Further, with respect to the region G1 between the set 301 of the region of interest and the set 302 of the region of interest, the other binarized pixel value (pixel value B indicating black) of the adjacent region of interest 30B and the adjacent region of interest 30C. The same pixel value B as the other binarized pixel value (pixel value B indicating black) is set.

In this way, a binary image 35 (FIG. (C)) in which the region of the object OB is represented by the pixel value W indicating white and the other region is represented by the pixel value B indicating black is generated. .. In this binary image 35, even when the shadow SD of the object OB is generated, only the area of the object OB can be represented by the pixel value W indicating white, and the area of the object OB can be displayed. The accuracy of identification can be improved.

As described above, in the present embodiment, first, the region of interest 30 including the feature points presumed to be the boundary (neighborhood) between the object OB and the background BG and having a predetermined width in the row direction is specified and roughly narrowed down. Then, detailed analysis is performed for each of the regions of interest 30.

In the conventional method (for example, the method described in Patent Document 1), the captured image 21 (and its grayscale image 22) of the object OB is scanned in the row direction (horizontal direction) (specifically, for example, the row). It detects (estimates) the edge of the object OB (the boundary between the object OB and the background BG) while detecting the brightness of one pixel in the direction), and when the shadow SD of the object OB occurs. There was a problem that the identification of the object OB was not sufficient. However, in the present embodiment, the position of only the object OB is specified even when a region having a lower brightness than the actual one is present on the captured image 21 due to the shadow SD of the object OB. It is possible to improve the accuracy of the operation.

Specifically, for a plurality of captured images 21, the correct answer rate when the object OB was specified by the conventional method was 51.8%, whereas the correct answer rate was 51.8%, whereas the object OB was specified by the present embodiment. The correct answer rate was 79.3%, and the accuracy could be greatly improved.

<Image processing method>
Next, the image processing method of the present embodiment will be described with reference to FIGS. 7 to 10. FIG. 7 is a flow chart showing an example of the processing flow of the image processing method according to the present embodiment.

In the image processing method of the present embodiment, first, the image acquisition process is performed in step S01. The image acquisition process is a process of acquiring a process target image 22 including an image of the object OB. In this process, for example, the captured image 21 is received from an external device or the like via the input means 5, or the captured image 21 stored in the storage means 3 is acquired. The captured image 21 is, for example, an image captured together with the object OB by arranging, for example, a white background bar BS on the background of the object OB (for example, a reinforcing bar). In this step, after converting the captured image 21 into a grayscale image, a necessary area is cut out to generate a processing target image 22. The area required here is, for example, a rectangular area including a background BG (a part of the background bar BS) and at least one object (reinforcing bar) OB, and is m in the row direction (horizontal direction on the paper surface, horizontal direction). This is an area in which a plurality of (n ≧ 2) pixels (n rows and m columns) are gathered in the column direction (vertical direction and vertical direction of the paper surface) (m ≧ 2) (see FIG. 2).

Alternatively, in this step, the processed target image 22 (processed above) is directly acquired from an external device or the like via the input means 5. Further, the processing target image 22 stored in the storage means 3 may be acquired.

In step S03 following step S01, an area of interest determination process is performed, in subsequent step S05, a threshold value determination process is performed, and in subsequent step S07, a generation process is performed. Details of the region of interest determination process, the threshold value determination process, and the generation process will be described later.

The region of interest determination process in step S03 shown in FIG. 7 will be described with reference to FIG. The figure is a flow chart showing an example of the flow of the region of interest determination process.

First, in step S301, the representative values of the pixels in each column direction are calculated for the image 22 to be processed. Specifically, from one end to the other end of the image 22 to be processed in the horizontal (horizontal) direction, the average brightness of a plurality of pixels arranged in the column direction (vertical direction, vertical direction) for each row of the m-th row pixel group 23. The value 24 is calculated, and the average brightness value 24 (vertical axis) with respect to the position in the row direction (horizontal axis) is graphed (see FIG. 3 (A)). This graph shows the change in the average luminance value 24 in the row direction.

In the following step S303, the luminance distribution threshold value 25 is obtained. Specifically, among the representative values (average luminance value 24) of the pixel group 23 of all the columns of the image to be processed 22 calculated in step S301, the maximum average luminance value 24MAX and the minimum average luminance value 24MIN are acquired. , The brightness value (average value) in the middle is calculated, and this is set as the brightness distribution threshold 25 (see FIG. 4 (D)).

In the following step S305, in the graphs of changes in the average luminance value (representative value) 24 arranged in the row direction (FIGS. 3 (B) and 4 (D)), an ON section (a section in which the brightness tends to be small) and an OFF section (a section in which the brightness tends to be small) ( A mask 27 for allocating (a section in which the brightness tends to be large) is created. Specifically, the intersection of the luminance distribution threshold value 25 and the average luminance value 24 is set as the feature point 26, and for example, the first feature point 26 (26A) and the second feature point 26 (26B) from the left along the row direction. Set one mask 27 (27A) with, and set one mask 27 (27B) with the third feature point 26 (26C) and the fourth feature point 26 (26D) (see FIG. 3B). ..

As shown by the alternate long and short dash line in FIG. 4D, the mask 27 is a rectangular region having a long side of a vertical line segment including two feature points 26 that are sequentially adjacent to each other from the left end side of the image to be processed 22. Theoretically, it is set for each object OB (except for the unintended feature point 26).

Next, in steps S307 to S317 of FIG. 8, overexposure correction processing is performed. This correction process is a process executed by the correction means 15 shown in FIG. 1 (B). For example, especially in the case of bar arrangement inspection at a construction site, the reinforcing bar OB, which is the object, is fixed in the irregular ambient light outdoors or indoors, and is taken in the optimum state for image processing. Is often difficult. Specifically, when the reinforcing bar OB is irradiated with strong sunlight or a camera flash, the surface of the reinforcing bar OB becomes bright (shining), and as a result, the inside of the object OB is unintentionally turned off. There may be a section. In this case, although the image should be processed as the object (reinforcing bar) O, there is a problem that the reinforcing bar OB is erroneously recognized as thinner than the actual object due to the occurrence of an unintended OFF section.

Therefore, in the overexposure correction process, a process is performed so that an unintended OFF section does not occur, that is, if an unintended feature point 26 exists, it is eliminated. As a result, even if the processing target image 22 is based on the image (captured image 21) taken in the irregular ambient light outdoors or indoors, the region of the target object OB can be recognized as the ON section. , The accuracy of identifying the region of the object OB is improved.

First, in step S307, the width (row direction, lateral direction, horizontal direction width, the same applies hereinafter) L3 of each OFF section is calculated in FIG. 3B. In the following step S309, the maximum width (ON section maximum width) L4 of the ON sections is calculated.

In the following step S311, the average value Av1 of the widths of all the OFF sections having a width L3 smaller than the maximum width L4 of the ON section (hereinafter, referred to as “small OFF section”) is calculated. In the following step S313, the average value Av2 and the standard deviation of the widths of all the OFF sections other than the small OFF section (the OFF section having the width L4 or more of the ON section having the maximum width L3, hereinafter referred to as "large OFF section"). Calculate with S2. In the following step S315, the average value Av3 and the standard deviation S3 of the widths of all the ON sections are calculated.

Then, in the following step S317, the following equation (1) is determined, and when the right side is larger than the left side, the small OFF section is replaced with the ON section.

Av2-S2-Av1> Av3 + S3-Av1 (Equation 1)

By this correction process, overexposure occurs on the object OB in the OFF section which is narrower than the actual background BG between the plurality of object OBs and narrower than the object OB (and its shadow). It is possible to presume that the area is covered and replace the overexposed area with the object OB (and its shadow).

In step S319 following these correction processes, the region of interest 30 is determined. Specifically, with the two feature points 26 of the mask 27 as the center, the width L2 of 1/2 of the width L1 in the row direction (horizontal direction) of the mask 27 is set in the row direction (horizontal direction) for the image 22 to be processed. The secured rectangular area is specified and determined as the area of interest 30 (see FIG. 4D). The unintended OFF section is corrected to the ON section by the above-mentioned correction process, the mask 27 is also provided corresponding to the corrected ON section, and the region of interest 30 is determined based on the mask 27. Further, the sets 301, 302, ... Of the related areas of interest are determined by combining the two areas of interest 30 that are sequentially arranged in the row direction from the left end of the image 22 to be processed, for example.

The threshold value determination process in step S05 shown in FIG. 7 will be described with reference to FIG. The figure is a flow chart showing an example of the flow of the threshold value determination process.

First, in step S501, a histogram of all the pixels in the region of interest 30 is calculated for each region of interest 30. In the following steps S503 to S511, the calculated histogram is smoothed (smoothed).

Hereinafter, an example of histogram smoothing processing will be specifically described. In step S503, the number of sections is set to 11, for example, and the moving average is performed several times (for example, 3 times) on the histogram calculated in step S501.

In the following step S505, the slope is calculated for each peak in the above histogram. Then, in step S507, the number of sections is set to 3, for example, and the moving average is performed on the slope, for example, once.

In the following step S509, the data sequence obtained as a result of finding the moving average of the slope is scanned in order from the left, and the middle of the points where the front is positive (+) and the back is negative (-) is found. Delete the data sequence of "0".

In step S511, the mountain (histogram) is drawn with the middle of the points where the front is positive (+) and the back is negative (−) as the apex (peak).

As a result, a histogram (smoothed) as shown in FIG. 5B is generated for each region of interest 30. The above smoothing process (for example, the number of moving average sections, the number of times, etc.) is an example.

In the following step S513, the binarization threshold is determined for each region of interest 30. Specifically, when there are a plurality of histogram peaks for each region of interest 30, the first peak is estimated as the object OB, and the second peak is estimated as the shadow SD. Then, the brightness value located at the midpoint CNT of the first peak PA1 and the second peak PK2 from the left (in this example, for example, 130 nt is determined as the binarization threshold value in the region of interest 30 (FIG. 5 (Fig. 5)). B) See).

Next, the region of interest determination process in step S07 shown in FIG. 7 will be described with reference to FIG. The figure is a flow chart showing an example of the flow of the generation process.

First, in step S701, in each region of interest 30, a binary image of the region of the target object OB and the region excluding the target object OB is generated based on the determined binarization threshold value. Specifically, first, each pixel in each region of interest 30 is binarized. That is, a pixel having a brightness value smaller than the binarization threshold value is defined as a pixel value W indicating white, and a pixel having a brightness value larger than the binarization threshold value is defined as a pixel value B indicating black.

After that, the noise in the region of interest 30 is removed. Specifically, in step S703, expansion processing is performed to increase the white region of the binary image, and in step S705, contraction processing is performed to decrease the white region.

In the following step S707, the inside of each region of interest 30 is estimated to be an object OB and binarized. Specifically, the pixel value W indicating white is set for the pixel in the ON section, and the pixel value B indicating black is used for the pixel in the OFF section, and the process is terminated.

<Image processing program>
The image processing program of the present embodiment is a program that causes a computer to execute the image processing method shown in FIGS. 7 to 10 described above.

Although one embodiment of the present invention has been described above, the image processing apparatus, image processing method, and image processing program according to the present invention are not limited to the above-described embodiments and do not deviate from the gist of the present invention. Of course, various changes can be made within the range.

For example, the region of interest 30 may be determined by learning. Specifically, a plurality of sample images (grayscale images) of the edge of a certain object OB and its vicinity are stored in a predetermined area of the storage means 3, and the processing target image 22 and the sample image are of interest by pattern matching. Region 30 may be determined.

Further, the representative value is not limited to the average luminance value of the pixel group 23 in the column direction, and may be a standard deviation value indicating variation in the luminance values of a plurality of pixels constituting the pixel group 23 in the column direction. May be a dispersion peak. Further, the feature point may be determined by combining the peak of the dispersion and the intersection of the above-mentioned luminance distribution threshold value 25 and the average luminance value 24.

Further, in addition to the correction of overexposure, for example, when the object OB is a reinforcing bar, a correction means for correcting discoloration due to rust may be provided. Although the rust is generated on the reinforcing bar, the brightness value tends to be biased toward the brighter (higher) side. Therefore, in the correction of rust, for example, it is advisable to check the histogram of R, G, and B in the captured image 21 and perform correction such as converting a portion having a large amount of red (R) to blue (B).

Further, the image processing device 10 may be configured to include an image pickup means, and may be, for example, a tablet terminal with a camera function. In that case, the captured image 21 may be acquired (photographed) by the imaging means, stored in the storage means 3, and processed into the processing target image 22.

As described above, in the present embodiment, the case where the processed target image 22 includes a plurality of target objects OB (reinforcing bars) whose axes are arranged along the vertical direction has been illustrated. However, the present invention is not limited to this, and the processing target image 22 may include at least one object OB (reinforcing bar) whose axis is arranged along the horizontal direction.

Specifically, the processing target image 22 of the present embodiment includes, for example, a plurality of object OBs arranged in a column direction (vertical direction) of the processing target image 22 shown in FIG. 2B rotated by 90 degrees. Image). In that case, the first arrangement direction (direction along the axis of the object OB) is the row direction (direction of the same row, horizontal direction of the paper surface, horizontal direction), and the second arrangement direction is the column direction (direction of the same column, paper surface). Vertical direction, vertical direction). Then, the region of interest determination means 13 calculates a representative value (average brightness value 24) of the pixels for each first arrangement direction (row direction), and sets the representative value at the feature point 26 in the second arrangement direction (column direction). Based on this, a set of regions of interest 30 arranged in the second arrangement direction for one object OB is determined.

As described above, the first arrangement direction of the present invention is the row direction (direction of the same row, horizontal direction of the paper surface, horizontal direction), and the second arrangement direction is the column direction (direction of the same column, vertical direction of the paper surface, vertical direction). Direction).

Further, the object OB is not limited to the reinforcing bar.

1 Control means 3 Storage means 5 Input means 9 Communication means 10 Image processing device 11 Image acquisition means 13 Interest area determination means 15 Correction means 17 Threshold determination means 19 Generation means 21 Captured image 22 Image to be processed (grayscale image)
23 Pixel group 24 Representative value (average brightness value)
24MAX Maximum average brightness value 24MIN Minimum average brightness value 25 Threshold 26 Feature point 27 Mask 30 Area of interest OB Object

Claims (17)

An image processing device that identifies an area of an object from an image of the object.
For the processing target image including a plurality of pixels in the first arrangement direction extracted from the image and the second arrangement direction orthogonal to the first arrangement direction, a representative value of the pixels is calculated for each of the first arrangement directions. An area of interest determining means for determining an area of interest for the object based on the feature points of the representative values in the direction of the second arrangement.
A threshold value determining means for determining a threshold value for each region of interest, and
A generation means for generating a binary image of a region of the object and a region excluding the object based on the threshold value, and
An image processing device characterized by having. The representative value is an average luminance value of pixels arranged in each of the first array directions.
The image processing apparatus according to claim 1. The region of interest determining means determines a set of regions of interest arranged in the second arrangement direction for one object.
The image processing apparatus according to claim 1 or 2. The generation means binarizes each pixel in each of the interest regions based on the threshold value, and the region sandwiched between the set of interest regions is combined with the binarized pixel value of the adjacent interest region. The binary image is generated by using the same pixel value.
The image processing apparatus according to claim 3. The threshold value determining means calculates a histogram in the region of interest and determines the threshold value based on the histogram.
The image processing apparatus according to any one of claims 1 to 4, wherein the image processing apparatus is characterized by the above. The threshold value determining means determines the luminance value located between the two vertices of the histogram as the threshold value.
The image processing apparatus according to claim 5. The region of interest determining means has a correction means for eliminating the unintended feature points.
The image processing apparatus according to any one of claims 1 to 6, wherein the image processing apparatus is characterized by the above. The image to be processed is a grayscale image.
The image processing apparatus according to any one of claims 1 to 7, wherein the image processing apparatus is characterized by the above. An image processing method for identifying an area of an object from an image obtained by photographing the object.
For the processing target image including a plurality of pixels in the first arrangement direction extracted from the image and the second arrangement direction orthogonal to the first arrangement direction, a representative value of the pixels is calculated for each of the first arrangement directions. A region of interest determination step for determining the region of interest for the object based on the feature points of the representative values in the direction of the second arrangement, and
A threshold determination step for determining a threshold value for each region of interest, and
A generation step of generating a binary image of a region of the object and a region excluding the object based on the threshold value, and
An image processing method characterized by having. The representative value is an average luminance value of pixels arranged in each of the first array directions.
The image processing method according to claim 9, wherein the image processing method is characterized by the above. A set of said regions of interest aligned in the second arrangement direction for one said object is determined.
The image processing method according to claim 9 or 10. Each pixel in each of the interest regions is binarized based on the threshold value, and the region sandwiched between the set of interest regions is set to the same pixel value as the binarized pixel value of the adjacent interest region. By doing so, the binary image is generated.
The image processing method according to claim 11, wherein the image processing method is characterized. A histogram in the region of interest is calculated, and the threshold value is determined based on the histogram.
The image processing method according to any one of claims 9 to 12, wherein the image processing method is characterized. A luminance value located between two vertices of the histogram is determined as the threshold value.
The image processing method according to claim 13, wherein the image processing method is characterized. Make corrections to eliminate unintended feature points.
The image processing method according to any one of claims 9 to 14, wherein the image processing method is characterized. The image to be processed is a grayscale image.
The image processing method according to any one of claims 9 to 15, characterized in that. An image processing program for causing a computer to execute the image processing method according to any one of claims 9 to 16.