JP6681077B2 - Labeling processor - Google Patents

Description

  The present invention relates to a labeling processing device that gives the same label to each successive pixel in image data, and more particularly to a labeling processing device that enables high-speed processing and low cost of a processing circuit.

In order to distinguish the area of connected pixels in the binary image from the area of other connected pixels, a labeling process is performed to attach different labels to each connected area. By performing the labeling process, it is possible to obtain the feature amount such as the area and the center of gravity of each connected region. Such image processing technology is used for defect inspection of products in various manufacturing plants.
As an example of the labeling process, a scan mask (reference range, FIG. 7, etc.) for checking the labels attached to the upper left, upper, upper right, and left pixels of the pixel of interest is set, and raster scanning starts from the upper left pixel of the image data. There is a labeling process in order. In the case of this labeling algorithm, when there are a plurality of pixels to which different labels are attached within the reference range, it is necessary to determine the priority order of how to label the pixel of interest. Depending on the set priority order, different labels may be given to pixels that are in the same connected area, so connection information indicating the connection relationship between the labels is generated during the labeling process. Then, it is necessary to connect the labels using the connection information after the raster scan is completed (Patent Document 1).
When utilizing an image captured from a camera for inspection on a surveillance camera or a manufacturing line, an image processing apparatus that executes a labeling process is required to perform high-speed data processing in real time. In Patent Document 2, by determining the boundary perimeter of a connected area and removing an area having a predetermined peripheral length or more or a connected area having a predetermined peripheral length or less, the labeling process can be completed by one raster scan, and real time is realized. The data processing in is realized.

JP-A-3-184185 JP-A-3-206573

According to the invention described in Patent Document 1, the memory capacity for holding the connection information can be suppressed to a low level by constantly updating the connection relationship of the labeled connected areas. According to this, the labeling process can be completed with one raster scan.
However, in Patent Document 1, when the number of the primary label assigned to the pixel exceeds the maximum primary label number, the processing ends. In order to deal with various images, it is necessary to secure a sufficient number of labels that can be given to pixels, and thus there is a problem that a large memory capacity of the RAM for storing the processing result must be secured. Further, in the method described in Patent Document 2, since the labeling process is speeded up by removing the region in which the perimeter of the connected region is equal to or larger than the predetermined value or smaller than the predetermined value, there is a problem that the types of images that can be processed are limited. was there.
The present invention is to solve the problems in the conventional labeling processing as described above, when performing the labeling processing of various images, the labeling processing can be executed at high speed, and the labeling processing results can be obtained. An object of the present invention is to provide a labeling processing device capable of reducing the capacity of a memory to be stored.

In order to solve the above-mentioned problem, the invention according to claim 1 refers to a label given to a reference pixel adjacent to a pixel of interest of image data, and gives a labeling processing unit for giving a label to the pixel of interest, By the labeling processing unit, the first labeling information storage unit that stores the labeling information related to the label assigned by the labeling processing unit, the second labeling information storage unit that stores the labeling information transferred from the first labeling information storage unit, and the labeling processing unit. When the number of labels related to the assigned label reaches the number of flash labels, the labeling information stored in the first labeling information storage unit is transferred to the second labeling information storage unit and the first labeling information storage unit is flushed. And a flash control unit for controlling the flash.
In the invention according to claim 2, when the first labeling information storage unit is flushed, the labeling processing unit shows a connection relationship between a label given before the flushing and a label given after the flushing. The preprocessing connection information is stored in the first labeling information storage unit.

According to a third aspect of the present invention, in the flash control unit, when the label number related to the label given by the labeling processing unit becomes the flash label number in the middle of the line of the image data, the labeling processing unit. The first labeling information storage unit is flushed when the processing of the line is completed.
The invention according to claim 4 is characterized in that an access speed of the first labeling information storage unit is faster than an access speed of the second labeling information storage unit.
The invention according to claim 5 is characterized in that the storage capacity of the first labeling information storage unit is smaller than the storage capacity of the second labeling information storage unit.

In the invention according to claim 6, the flash label number is a storage capacity relating to labeling information corresponding to the flash label number, and the number of labels given by the labeling processing unit is the flash label number in the middle of a line. After that, the capacity obtained by adding the storage capacity of the labeling information corresponding to the predicted value of the number of labels newly assigned to the remaining part of the line does not exceed the storage capacity of the first labeling information storage section. It is set to.
In the invention according to claim 7, the labeling processing unit, as the labeling processing result, connection information indicating a connection relationship between labels, and additional information indicating the number and existence range of labeled pixels for each label. And storing it in the first labeling information storage unit, and generating pixel / label information indicating a correspondence relationship between each pixel in the image data and a label assigned to each pixel to generate the second labeling information. It is characterized in that it is stored in the storage unit.

  According to the present invention, when performing labeling processing of various images, the labeling processing can be executed at high speed, and the capacity of the memory for storing the labeling processing result can be reduced.

It is a hardware block diagram of the image processing apparatus which concerns on one Embodiment of this invention. It is a functional block diagram which shows the schematic structure of the labeling calculation process part which concerns on one Embodiment of this invention. FIG. 3 is a functional block diagram showing a configuration related to a labeling function of the image processing apparatus according to the embodiment of the present invention. It is a figure which shows an example of connection information. It is a figure which shows an example of additional information. It is a figure which shows an example of the binary image data used as the target of a labeling process. It is a figure which shows an example of the scanning mask used for a labeling process. It is a figure which shows an example of a labeling process, (a) is a figure which shows a process image, (b) is a figure which shows additional information, (c) is a figure which shows connection information. It is a figure explaining labeling processing, (a) is a figure which shows a processed image, (b) is a figure which shows additional information, (c) is a figure which shows the present process connection information, (d). FIG. 4 is a diagram showing preprocessing connection information. It is a figure explaining the labeling process after flash, (a) is a figure which shows a process image, (b) is a figure which shows additional information, (c) is a figure which shows the present process connection information, (D) is a figure which shows pre-processing connection information. It is a figure which shows an example of the process image after the labeling process by the present line process part is completed. (A)-(c) is a figure which shows an example of the data stored in the all connection information storage part of a 2nd labeling information storage part, and an additional information storage part. It is a figure which shows the outline of the address map of a 2nd labeling information storage part. It is a figure which shows an example of the processed image after label connection processing.

  Hereinafter, the present invention will be described in detail with reference to the embodiments shown in the drawings. However, the constituent elements, types, combinations, shapes, relative arrangements, and the like described in this embodiment are merely explanatory examples, not the gist of limiting the scope of the present invention thereto, unless specifically stated. .

[Hardware configuration]
FIG. 1 is a hardware configuration diagram of an image processing apparatus according to an embodiment of the present invention.
The image processing apparatus (labeling processing apparatus) 1 includes a CPU (Central Processing Unit) 11, a labeling calculation processing unit 13, a memory 15, and an I / F unit 16.
The CPU 11 has a register 12 and controls the entire image processing apparatus 1.
The labeling calculation processing unit 13 is an integrated circuit (IC) that executes various types of labeling-related processing according to instructions from the CPU 11. The labeling calculation processing unit 13 gives a temporary label (index) to the connected pixel area of the input image data. The labeling calculation processing unit 13 includes, for example, an SRAM (Static Random Access Memory) 14 that is a volatile storage unit as a storage unit, and temporarily stores the labeling information obtained by the labeling process in the SRAM 14.
The memory 15 has a storage capacity larger than that of the storage unit (SRAM 14) provided in the labeling calculation processing unit 13, but has a relatively low access speed. The memory 15 is composed of a volatile storage unit such as a DRAM (Dynamic Random Access Memory).
The I / F unit 16 is an interface connected to an external device. The I / F unit 16 is connected to a surveillance camera that supplies image data to be processed, another device that processes the image data processed by the labeling calculation processing unit 13, and the like.

[Schematic Configuration of Labeling Arithmetic Processing Unit]
FIG. 2 is a functional block diagram showing a schematic configuration of the labeling calculation processing unit according to the embodiment of the present invention.
The labeling calculation processing unit 13 includes a binarization processing unit 17, a noise removal unit 18, and a labeling processing unit 20.
The binarization processing unit 17 executes a process of binarizing a video signal (multi-valued image data) input from an external device such as a surveillance camera. The noise removing unit 18 removes noise from the image data by performing smoothing processing on the binarized image data. The labeling processing unit 20 temporarily assigns the same label (index) to the connected component of each pixel in the image data after noise removal, and outputs the result of the labeling processing.

[Functional configuration]
FIG. 3 is a functional block diagram showing the configuration related to the labeling function of the image processing apparatus according to the embodiment of the present invention.
The image processing apparatus 1 includes a control unit 19, a labeling calculation processing unit 13, a second labeling information storage unit 50, and a label concatenation processing unit 60.

The control unit 19 is realized by the CPU 11 shown in FIG. 1, and outputs a control signal for instructing the labeling calculation processing unit 13 to start the labeling process, a set value of the number of labels, and the like.
The labeling calculation processing unit 13 includes a labeling processing unit 20, a first labeling information storage unit 30, and a flash control unit 40. The labeling processing unit 20 executes a labeling process of giving a temporary label to each pixel in the binary image data after noise removal input from the noise removal unit 18 illustrated in FIG. 2.
The labeling processing unit 20 refers to the label attached to the reference pixel adjacent to the pixel of interest of the image data, and executes the process of attaching a label to the pixel of interest. The labeling processing unit 20 has a current line processing unit 21 and a 1-line preprocessing result holding unit 22.
The current line processing unit 21 detects the connection relationship between the pixel of interest and other pixels located around the pixel of interest. More specifically, the current line processing unit 21 performs a labeling process for assigning the same index (label) as a pixel located in 4-connected (4 vicinity) or 8-connected (8 vicinity) to a pixel of interest in image data. Run. The current line processing unit 21 generates labeling information including, as a result of the labeling process, connection information indicating a connection relationship between labels and additional information indicating the number and existence range of labeled pixels for each label. The first labeling information storage unit 30 is stored. The current line processing unit 21 also generates pixel / label information indicating the correspondence between each pixel in the image data and the label assigned to each pixel as a result of the labeling process, and the second labeling information storage unit 50. To memorize.
The one-line pre-processing result holding unit 22 is a unit that holds one line of pixel / label information generated as a result of the labeling process by the current line processing unit 21. The pixel / label information is information indicating which label is given to which pixel. The 1-line pre-processing result holding unit 22 is composed of a line buffer, and a plurality of bits are assigned to the area corresponding to each pixel of the line buffer. It is possible to sequentially label each pixel with the allocated number of bits as an upper limit.

  The first labeling information storage unit 30, as shown as the SRAM 14 in FIG. 1, is composed of a storage unit capable of high speed access. The first labeling information storage unit 30 stores labeling information related to the label given by the labeling processing unit 20. The labeling information generated by the labeling processing unit 20 includes connection information and additional information. The details of the connection information and the additional information will be described later. The first labeling information storage unit 30 is flushed (initialized) every time the number of labels given by the labeling processing unit 20 reaches a predetermined number. Therefore, the first labeling information storage unit 30 has a current processing connection information storage unit 31 and a preprocessing connection information storage unit 32 as means for storing the connection information (FIG. 4). The current processing connection information storage unit 31 stores connection information indicating a connection relationship between labels provided in the current labeling process. The preprocessing connection information storage unit 32 shows the connection relationship between the label given in the current labeling process and the label given in the previous labeling process (the labeling process immediately before flushing the first labeling information storage unit 30). Stores the connection information shown. Further, the first labeling information storage unit 30 has a first additional information storage unit 33 as a means for storing the additional information (FIG. 5).

When the number of labels provided by the current line processing unit 21 reaches a predetermined number of labels (the number of flash labels), the flash control unit 40 uses the labeling information stored in the first labeling information storage unit 30 as the second labeling information. A process for flushing the first labeling information storage unit 30 is executed while the data is transferred to the information storage unit 50. The flash control unit 40 includes a label number determination unit 41 and a controller 42.
The label number determination unit 41 inputs the number of labels assigned by the current line processing unit 21, and flashes the first labeling information storage unit 30 to the controller 42 when the number of labels reaches a predetermined flash label number. Output the request. The control unit 19 sets the number of flash labels, which is the standard of the flash request, in the label number determination unit 41. The control unit 19 may set the number of flash labels based on the storage capacity of the first labeling information storage unit 30, or based on the numerical value input from the outside via the I / F unit 16 shown in FIG. You may set it.
Here, the storage of the connection information and the additional information requires a capacity corresponding to the number of labels given by the labeling processing unit. The control unit 19 sets the number of labels smaller than the maximum number of labels of the labeling information that can be stored in the first labeling information storage unit 30 as the number of flash labels in the label number determination unit 41.

  When the flush request is input from the label number determination unit 41, the controller 42 reads the labeling information stored in each unit of the first labeling information storage unit 30 and writes it to the corresponding units of the second labeling information storage unit 50. Forward. Further, the controller 42 clears (flashes) the data in the first labeling information storage unit 30 by sequentially writing zero into the first labeling information storage unit 30.

The second labeling information storage unit 50 is realized by the memory 15 shown in FIG. The second labeling information storage unit 50 stores the labeling information transferred from the first labeling information storage unit 40. That is, the second labeling information storage unit 50 stores the connection information and the connection information read from the first labeling information storage unit 30 before the first labeling information storage unit 30 is flushed and after the labeling process for one frame image is completed. It is a means for storing and accumulating additional information.
Specifically, the second labeling information storage unit 50 stores all connections that store the respective connection information read from the current processing connection information storage unit 31 and the preprocessing connection information storage unit 32 of the first labeling information storage unit 30. The information storage unit 51 and the second additional information storage unit 52 that stores the additional information read from the first additional information storage unit 33 are provided. The second labeling information storage unit 50 also includes a pixel / label information holding unit 53 that holds the pixel / label information output from the labeling processing unit 20. The second labeling information storage unit 50 has an access speed lower than that of the first labeling information storage unit 30, but is cheaper than the first labeling information storage unit 30 and has a large capacity storage means. For example, the first labeling information storage unit 30 can be composed of SRAM, and the second labeling information storage unit 50 can be composed of DRAM.
By providing the second labeling information storage unit 50 in addition to the first labeling information storage unit 30, the current line processing unit 21 is secured while securing the storage capacity required for storing the labeling information generated from the image data of one frame. It is possible to reduce the capacity of the first labeling information storage unit 30 that stores the labeling information generated by the device in real time at high speed, and to reduce the cost of the image processing apparatus 1.

  The label connection processing unit 60 is means for performing label connection processing from all connection information, additional information, and pixel / label information stored in the second labeling information storage unit 50. When the pixels to which different labels are attached are connected to each other, the label connection processing unit 60 executes a process of reassigning the same label to each pixel forming one connection area. The label connection algorithm used by the label connection processing unit 60 is not particularly limited. The label concatenation processing unit 60 may be a circuit connected to the CPU 11 via a bus or an external device connected to the I / F unit 16.

[Connection information]
FIG. 4 is a diagram showing an example of connection information.
The connection information is information indicating the connection relationship between the labels. When different labels are connected to each other, a plurality of pixels with the labels form one connection region. The numbers 1 to 16 on the vertical axis and the horizontal axis represent the labels. In the illustrated example, the vertical axis represents the label of the pixel of interest, and the horizontal axis represents the label of the reference pixel. The illustrated example shows the initial state of the connection information, and only the portion where the connection information does not occur, that is, the label of the pixel of interest and the label of the reference pixel are the same is colored.

〔Additional information〕
FIG. 5 is a diagram showing an example of additional information. The additional information is information indicating in what range and in what size a set of pixels with a certain label exists in the image data, and is represented in a table format. The additional information is newly created every time the first labeling information storage unit 30 is flushed. The meaning of each title is as follows.
ID: identification information for identifying each additional information created for each flash. For example, when the ID “0” is added at the start of the labeling process, the ID is incremented every time the first labeling information storage unit 30 is flushed.
domain: information indicating the range of image data indicated by the additional information corresponding to the ID. When the flashing of the first labeling information storage unit 30 is executed in line units of image data, the information is information for specifying a processing line of image data.
index: A label given by the labeling process.
area: The number of pixels to which the label shown in the index is attached.
Xmin: X coordinate minimum value of the pixel area to which the label shown in index is attached.
Ymin: Y coordinate minimum value of the pixel area to which the label shown in index is attached.
Xmax: The maximum X coordinate value of the pixel area to which the label shown in index is attached.
Ymax: The maximum Y coordinate value of the pixel area to which the label shown in index is attached.

[Image example]
FIG. 6 is a diagram showing an example of the binary image data that is the target of the labeling process. In this figure, ruled lines are shown between the pixels in order to facilitate understanding of the positions of the pixels in the X and Y directions. The labeling processing unit 20 gives a label to a pixel of interest while moving the pixel of interest to be processed from the upper left of the image of one frame to the right in the drawing.

[Scan mask]
FIG. 7 is a diagram showing an example of a scanning mask used in the labeling process.
The scanning mask 70 is a scanning mask for so-called 8-connection (near 8), and the labeling processing unit 20 (FIG. 3) has four reference pixels 72a to 72d located on the upper left, upper, upper right, and left of the target pixel 71. It is shown that the label is given to the pixel 71 of interest with reference to the label attached to the pixel. Hereinafter, the reference pixels 72a to 72d may be collectively referred to as a "reference range". It should be noted that the numbers 1 to 4 attached to the reference pixels 72a to 72d indicate the order of priority of the labels to be assigned to the pixel of interest 71 when the scan mask 70 has a plurality of pixels to which labels are assigned. .

[Basic labeling procedure]
The labeling process will be described below with reference to FIG.
Binary image data for one frame as shown in FIG. 6 is supplied to the current line processing unit 21 of the labeling processing unit 20 pixel by pixel. The current line processing unit 21 refers to the labels attached to the reference pixels 72a to 72d in the scan mask 70 (FIG. 7) and sequentially performs the process of assigning labels to the target pixel 71.

FIG. 8 is a diagram showing an example of the labeling process, (a) is a diagram showing a processed image, (b) is a diagram showing additional information, and (c) is a diagram showing connection information. In the figure, the gray pixels indicate processed pixels. White and black solid pixels indicate unprocessed pixels, and white pixels among them indicate pixels to be labeled. Further, the arrow pointing to the right in the figure indicates the direction of processing. Further, this figure shows a state in which the labeling process is performed up to the middle of the sixth line.
Pixels with the labels “2” and “3” already exist in the scan mask 70, but the label “3” is given to the target pixel 71 according to the priority order of the pixels referred to by the scan mask 70 (FIG. 5). To be done.
As indicated by the broken line, the 1-line pre-processing result holding unit 22 holds the pixel / label information for one line from the pixel two pixels before the pixel of interest. The pixel / label information of the reference pixel located one line before the line being processed is read from the one-line preprocessing result holding unit 22 and referred to.
The current line processing unit 21 has a register that holds the pixel / label information of the pixel processed immediately before the pixel of interest 71, and refers to this at the time of the labeling process of the pixel of interest 71. After performing the labeling process on the pixel of interest 71, the current line processing unit 21 outputs the pixel / label information of the pixel located immediately before the pixel of interest 71 to the 1-line preprocessing result holding unit 22 and the second labeling information storage unit. It is stored in the pixel / label information holding unit 53 of 50.
The current line processing unit 21 stores the pixel / label information in the register in the 1-line preprocessing result holding unit 22 and the pixel / label information holding unit 53 at the end of the processing of one line, and clears the register.

<Additional information>
The current line processing unit 21 executes a process of updating the "domain" of the additional information (FIG. 8B) at the start of the process of one line. Further, the current line processing unit 21 executes a process of updating the additional information when a label is attached to the pixel of interest. Specifically, the current line processing unit 21 reads the additional information stored in the first additional information storage unit 33 of the first labeling information storage unit 30, and relates to the “domain” or the label given to the target pixel. A process of generating the additional information with the changed information and writing the additional information in the first additional information storage unit 33 is executed.
The additional information shown in FIG. 8B indicates the following state after the labeling process of the 14th pixel of the 7th line is completed.
(1) There are 12 pixels to which index1 is added, and the X axis minimum value is 1, the Y axis minimum value is 2, the X axis maximum value is 4, and the Y axis maximum value is 5.
(2) There are three pixels to which index2 is added, and the X axis minimum value is 14, the Y axis minimum value is 5, the X axis maximum value is 16, and the Y axis maximum value is 5.
(3) There are two pixels to which index3 is added, and the X axis minimum value is 12, the Y axis minimum value is 6, the X axis maximum value is 13, and the Y axis maximum value is 6.

<Connection information>
The current line processing unit 21 executes a process of updating the connection information (FIG. 8C) when a new label is added or when a new connection relationship between different labels is detected. Specifically, the current line processing unit 21 reads out the connection information currently stored in the processing connection information storage unit 31 and generates connection information in which the information related to the label connected to the label attached to the pixel of interest is changed. Then, the process of writing the current process connection information storage unit 31 is executed.
In the connection information of FIG. 8C, the vertical axis shows the target pixel and the horizontal axis shows the information of the reference pixel connected to the target pixel. As shown in FIG. 8A, when the 13th pixel on the 6th line is the target pixel, the target pixel is given the label “3”, but the pixel with the label “2” is added within the reference range. Since there is also, the information indicating that the pixels having the label “3” and the pixel having the label “2” are connected is added to the connection information. Specifically, the labeling processing unit 20 adds the information of the third row, second column (* mark) portion to the connection information shown in FIG. 8C.

[Flash processing]
Hereinafter, the flash of the first labeling information storage unit 30 will be described with an example in which the number of flash labels is four.
9A and 9B are diagrams for explaining the labeling process, FIG. 9A is a diagram showing a processed image, FIG. 9B is a diagram showing additional information, and FIG. 9C is a diagram showing current processing connection information. , (D) are diagrams showing preprocessing connection information. FIG. 9 shows a state in which the labeling process for the seventh line has been completed.

The first labeling information storage unit 30 is flushed when the number of labels given by the current line processing unit 21 reaches the number of flush labels.
In this example, the controller 42 determines that the number of labels related to the label given by the current line processing unit 21 becomes the number of flash labels in the middle of the line of the image data, and the labeling process of the line being processed by the current line processing unit 21 is completed. In this case, the first labeling information storage unit 30 is controlled to flash.

The specific control procedure in the flash process is as follows.
The current line processing unit 21 outputs information on the number of labels added in the current process to the label number determination unit 41 when a new label is attached to the pixel of interest. For example, the current line processing unit 21 notifies the label number determination unit 41 of the used label number “4” when the label “4” is newly added to the seventh line and the tenth pixel. The label number determination unit 41 determines whether or not the number of used labels reaches the number of flash labels, and when the number of used labels reaches the number of flash labels, outputs a flash request to the controller 42.
The current line processing unit 21 outputs a line processing end notification to the controller 42 every time the processing of one line is completed.
When the controller 42 receives the flush request from the label number determination unit 41 and the line processing end notification from the current line processing unit 21, the controller 42 transfers the labeling information from the first labeling information storage unit 30 to the second labeling information storage unit. Transfer to 50. That is, the controller 42 reads the current processing connection information stored in the current processing connection information storage unit 31 and the preprocessing connection information stored in the preprocessing connection information storage unit 32 to store the second labeling information. The process of storing in the all connection information storage unit 51 of the unit 50 is executed. In addition, the controller 42 reads the additional information stored in the first additional information storage unit 33 and stores it in the second additional information storage unit 52 of the second labeling information storage unit 50.
After that, the controller 42 executes a process of flushing the first labeling information storage unit 30.
Further, the controller 42 instructs the current line processing unit 21 to initialize the label.

[Labeling after flash]
FIG. 10 is a diagram for explaining the labeling process after flushing, (a) is a diagram showing a processed image, (b) is a diagram showing additional information, and (c) is current processing connection information. It is a figure and (d) is a figure which shows pre-processing connection information. FIG. 10 shows a state where the labeling process has been performed up to the middle of the eighth line.
After the first labeling information storage unit 30 is flushed, the current line processing unit 21 initializes the label to be given to the pixel according to the label initialization command from the controller 42. That is, the current line processing unit 21 gives the label “1” to the pixel of interest to which the label is first detected after the flash. The current line processing unit 21 generates additional information by counting up the ID of the additional information in the pre-processing, and stores it in the first additional information storage unit. At this time, the current line processing unit 21 gives "domain" indicating the processing line information regarding the processing line after the flush.
The current line processing unit 21 updates the current processing connection information (FIG. 10 (c)) when a new label is added or when a new connection relationship between different labels is detected in the line processed after flushing. Then, it is stored in the current processing connection information storage unit 31.
When the current line processing unit 21 detects the connection relationship between the label in the line processed before the flush and the label in the line processed after the flush, the preprocess connection information (FIG. 10D). Is updated and stored in the preprocessing connection information storage unit 32.

FIG. 11 is a diagram showing an example of a processed image after the labeling process by the current line processing unit is completed. This figure shows the information finally stored in the pixel / label information holding unit 53 of the second labeling information storage unit 50.
The current line processing unit 21 outputs a frame processing end notification to the controller 42 when the processing of the image data of one frame (the processing of the 23rd pixel on the 23rd line in the figure) is completed.
The controller 42 receives the frame processing end notification from the current line processing unit 21 and transfers all the information stored in the first labeling information storage unit 30 to the second labeling information storage unit 50; A process of flushing the labeling information storage unit 30 is executed.

  12A to 12C are diagrams showing an example of data stored in the all connection information storage unit and the additional information storage unit of the second labeling information storage unit. This figure shows the information stored in the second labeling information storage unit when the labeling process is completed for one frame image. (A) shows the labeling information of ID0, (b) shows the labeling information of ID1. (C) shows the labeling information of ID2. Each piece of labeling information includes additional information, current processing connection information, and preprocessing connection information.

  FIG. 13 is a diagram showing an outline of the address map of the second labeling information storage unit. In this figure, only the address maps relating to the all connection information storage unit 51 and the second additional information storage unit 52 are shown. The second labeling information storage unit 50 is formed with storage areas 54a to 54c for sequentially storing labeling information (FIGS. 12A to 12C) for each ID. The respective labeling information storage areas are an additional information storage area 55 for storing additional information, a current processing connection information storage area 56 for storing current processing connection information, and a preprocessing connection information storage area 57 for storing preprocessing connection information. Is equipped with.

[Label connection processing]
The label concatenation processing unit 60 (FIG. 3) stores the pixel / label information (FIG. 11) stored in the pixel / label information holding unit 53 of the second labeling information storage unit 50 and the all connection information storage unit 51. A process of connecting the labels is executed using the connection information (FIG. 12).
The label concatenation processing unit 60 sequentially confirms the pixel / label information (FIG. 11) in the pixel / label information holding unit 53 of the second labeling information storage unit 50 from the 0th line, and concatenates the labels.
First, the label connection processing unit 60 uses the current processing connection information of ID “0” (FIG. 12A) stored in the all connection information storage unit 51 of the second labeling information storage unit 50 to perform label connection processing. To execute. The label concatenation processing unit 60 determines that the label (index) “2” is connected to the labels “3” and “4” based on the current processing connection information, and the labels “3” and “4” are allocated. The labels of the selected pixels are all changed to "2". The label concatenation processing unit 60 temporarily stores information indicating that the label “4” has been changed to the label “2”.

Next, the label concatenation processing unit 60 executes the label concatenation processing using the current processing connection information and preprocessing connection information of ID "1" (FIG. 12B).
The preprocessing connection information includes the information that the label “1” is connected to the label “4” in the preprocessing, and the label concatenation processing unit 60 determines that the label “4” is identified in the label concatenation processing of ID “0”. Since the information indicating that the label has been changed to “2” is stored, the label concatenation processing unit 60 changes the label “1” in the ID “1” to the label “2”.
The label concatenation processing unit 60 labels the independent labels “2”, “3”, and “4” that are not connected to other labels in order from the smallest unused label to the labels “3”, “4”, and “5”, respectively. Re-roll. Further, the label concatenation processing unit 60 temporarily stores information indicating that the labels “1” to “4” with the ID “1” have been changed to the labels “2” to “5”, respectively.

Finally, the label concatenation processing unit 60 executes the label concatenation processing in the same manner as above using the current processing connection information and preprocessing connection information of ID "2" (FIG. 12 (c)).
FIG. 14 is a diagram showing an example of a processed image after the label connecting process. Finally, the label connected image as shown in FIG. 14 is obtained.

[Modifications, etc.]
In the present embodiment, the label number determination unit 41 sets the number smaller than the maximum label number of the labeling information that can be stored in the first labeling information storage unit 30 as the flash label number. Specifically, it is set as follows. That is, the number of flash labels is the storage capacity required for storing the labeling information corresponding to the number of flash labels, and the number of labels given by the current line processing unit 21 becomes the number of flash labels in the middle of the line, The storage capacity required to store the labeling information corresponding to the predicted value of the number of labels newly added to the remaining part of the line is set so that the added capacity does not exceed the storage capacity of the first labeling information storage section 30. To be done. Even if the number of labels assigned by the current line processing unit 21 exceeds the number of flash labels during the processing of one line, the maximum number of labels is stored in the first labeling information storage unit 30 until the processing of one line is completed. The labeling information can be sequentially stored within the range.

  In the present embodiment, the flushing of the first labeling information storage unit 30 is executed after the processing of one line is completed, but it may be configured to flush in the middle of the line. In this case, the maximum number of labels of the labeling information that can be stored in the first labeling information storage unit 30 can be set in the label number determination unit 41 as the number of flash labels. The “domain” of the additional information (FIG. 5) becomes information of the starting point pixel and the ending point pixel of the area divided by the flash process. The controller 42 controls to flash the first labeling information storage unit 30 when the number of labels provided by the labeling processing unit 20 becomes the number of flash labels and the pixels with the labels are not continuous.

In the above embodiments, the SRAM has been described as an example of a storage unit having a small capacity but a high access speed, and the DRAM has been described as an example of a storage unit (memory) having a large capacity but a low access speed. In the above, the storage means used for the first labeling information storage unit 30 and the second labeling information storage unit 50 is not limited to this.
For example, a flip-flop circuit can be used in the first labeling information storage unit 30 as a storage device with high high-speed response other than SRAM. As the second labeling information storage unit 50, a flash memory or an HDD (Hard Disk Drive) can be used instead of the DRAM. The second labeling information storage unit 50 may be an external memory (external storage device) connected via the I / F unit 16.
In the above embodiment, the relation between the access speed and the capacity is defined when describing the SRAM or the DRAM. However, since a storage device having a high access speed is generally expensive, the device is provided at low cost. It is preferable to use a small-capacity storage device, while the storage device with a low access speed is often described as an inexpensive storage device despite its large storage capacity. It does not limit the invention in any way.

  In the present embodiment, when the first labeling information storage unit 30 is flushed, the label given by the current line processing unit 21 is also initialized, but the current line processing unit 21 continues to the label given before the flushing even after the flushing. A label may be given. That is, the current line processing unit 21 may count up and add the label regardless of whether the flush is executed.

[Usage of labeling equipment]
The image processing apparatus (labeling processing apparatus) according to the present embodiment is incorporated into a defect inspection apparatus in a manufacturing line, a surveillance camera apparatus for performing object tracking processing, an inspection apparatus in the medical field (for example, an ophthalmic inspection apparatus), and the like. You can By performing a labeling process on the image data, it is possible to specify the connected area existing in the input image, obtain the area and the center of gravity of the specified connected area, and use the method for various inspections.
If it is sufficient to know the existing area of the connected area and the size of the connected area (the number of pixels forming the connected area), the additional information can be mainly used and the label connecting process is necessary. It may not be done.

[Summary of Example Embodiments of the Present Invention and Actions and Effects]
<First embodiment>
The labeling processing device (image processing device 1) according to the present aspect refers to a label attached to a reference pixel adjacent to a pixel of interest of image data, a labeling processing unit 20 that attaches a label to a pixel of interest, and a labeling process. The first labeling information storage unit 30 that stores the labeling information related to the label given by the copy unit, the second labeling information storage unit 50 that stores the labeling information transferred from the first labeling information storage unit, and the labeling processing unit. When the number of labels related to the assigned label reaches the number of flash labels, the labeling information stored in the first labeling information storage unit is transferred to the second labeling information storage unit and the first labeling information storage unit is flushed. And a flash control unit 40 for controlling the flash.

A high-speed operation is required for a memory that directly stores the labeling information generated by the labeling processing unit in real time. However, if all the labeling information generated from one frame of image data is to be stored in a memory that can operate at high speed, the storage capacity required for the memory becomes large and the device cost becomes excessive. is there.
Therefore, in the labeling processing apparatus according to this aspect, a plurality of memories (first labeling information storage section and second labeling information storage section) for storing the labeling processing result are arranged. The first labeling information storage unit is a means for directly storing the labeling information generated by the labeling processing unit in real time, and is a memory that requires high-speed operation. The second labeling information storage unit is a unit that temporarily saves the labeling information stored in the first labeling information storage unit from the first labeling information storage unit at a predetermined timing. It is a memory whose capacity is required. In this way, by preparing a plurality of labeling information storage units, it becomes possible to give each labeling information storage unit a required performance according to its role.
Therefore, according to this aspect, the labeling process can be executed at high speed when the labeling process of various images is performed, and the capacity of the memory for storing the labeling process result in real time can be reduced. That is, according to this aspect, while maintaining the high speed and the real-time property of the image processing apparatus, even with an inexpensive apparatus configuration, a sufficient capacity for storing the generated one frame of labeling information is secured. can do.

<Second embodiment>
In the labeling processing apparatus (image processing apparatus 1) according to this aspect, the labeling processing unit 20 is configured such that, when the first labeling information storage unit 30 is flushed, the label given before the flush and the label given after the flush. It is characterized in that preprocessing connection information indicating a connection relationship with is stored in the first labeling information storage unit 30.
According to this aspect, even if the first labeling information storage unit is flushed, the connection relationship between the labels attached before and after the flashing is stored in the first labeling information storage unit, so that the labels before and after the flushing will not be a problem. A connection process can be performed.

<Third embodiment>
In the labeling processing device (image processing device 1) according to this aspect, the flash control unit 40, when the label number related to the label given by the labeling processing unit 20 becomes the flash label number in the middle of the line of the image data, It is characterized in that the first labeling information storage unit 30 is flushed when the processing of the line by the labeling processing unit is completed.
Although it is possible to flash the first labeling information storage part in the middle of the line of the image data, there is a problem that the labeling process becomes complicated and the circuit related to the labeling process becomes enlarged.
According to this aspect, by performing the flushing of the first labeling information storage unit on a line-by-line basis, the labeling process can be simplified, and the circuit related to the labeling process can be made compact.

<Fourth Embodiment>
The labeling processing device (image processing device 1) according to this aspect is characterized in that the access speed of the first labeling information storage unit 30 is faster than the access speed of the second labeling information storage unit 50.
According to this aspect, the labeling information generated by the labeling processing unit can be stored in the first labeling information storage unit at high speed in real time. Further, since the memory having a relatively low access speed is inexpensive, even if the capacity of the second labeling information storage unit is increased to the extent necessary to store the labeling information for one frame, the device can be configured inexpensively. be able to.

<Fifth Embodiment>
The labeling processing apparatus (image processing apparatus 1) according to this aspect is characterized in that the storage capacity of the first labeling information storage unit 30 is smaller than the storage capacity of the second labeling information storage unit 50.
According to this aspect, even if the first labeling information storage unit is a memory that can be accessed at high speed, the storage capacity can be made low, so that the cost of the device can be reduced. Further, by increasing the capacity of the second labeling information storage unit, even if the capacity of the first labeling information storage unit is reduced, the number of labels to be given in the labeling process is not limited, and thus the labeling process of various images is performed. It becomes possible to do.

<Sixth embodiment>
In the labeling processing apparatus (image processing apparatus 1) according to the present aspect, the number of flash labels is the storage capacity of the labeling information corresponding to the number of flash labels and the number of labels given by the labeling processing unit 20 in the middle of the line. After the number of labels is reached, the storage capacity of the first labeling information storage unit 30 is calculated by adding the storage capacity of the labeling information corresponding to the predicted value of the number of labels newly added to the rest of the line to the storage capacity of the first labeling information storage unit 30. The feature is that it is set not to exceed.
If the flashing is performed in the middle of the line of the image data, the labeling information can be stored in the first labeling information storage unit up to the maximum storage capacity. However, considering the simplification of the labeling process and the compactness of the circuit related to the labeling process, it is preferable to perform the flash in units of lines. For that purpose, it is necessary to ensure the flash in units of lines.
In this aspect, even if the number of labels given by the labeling processing unit reaches the number of flash labels in the middle of the line, the labeling information generated in the first labeling information storage unit at least until the labeling process of the line is completed. It also has a capacity to store. By doing so, it is possible to guarantee the flash processing for each line.

<Seventh embodiment>
In the labeling processing device (image processing device 1) according to this aspect, the labeling processing unit 20 labels, as the labeling processing result, the connection information indicating the connection relationship between the labels, the number of labeled pixels, and the existing range. The additional information shown for each pixel is generated and stored in the first labeling information storage unit 30, and the pixel / label information indicating the correspondence between each pixel in the image data and the label assigned to each pixel is generated to generate the pixel / label information. It is characterized in that it is stored in the two-labeling information storage unit 50.
According to this aspect, the connection information and the additional information that are changed in real time during the labeling process are stored in the first labeling information storage unit, and the pixel / label information that is not changed during the labeling process is stored in the second labeling information storage unit. Therefore, the storage capacity of the first labeling information storage section can be reduced.

  DESCRIPTION OF SYMBOLS 1 ... Image processing device (labeling processing device), 11 ... CPU, 12 ... Register, 13 ... Labeling calculation processing part, 14 ... Line, 15 ... Memory, 16 ... I / F part, 17 ... Binarization processing part, 18 ... noise removing unit, 19 ... control unit, 20 ... labeling processing unit, 21 ... current line processing unit, 22 ... line preprocessing result holding unit, 30 ... first labeling information storage unit, 31 ... current processing connection information storage unit, 32 ... Preprocessing connection information storage unit, 33 ... First additional information storage unit, 40 ... Flash control unit, 41 ... Label number determination unit, 42 ... Controller, 50 ... Second labeling information storage unit, 51 ... All connection information storage 52, second additional information storage unit, 53, pixel / label information holding unit, 54a to 54c, storage area, 55, additional information storage area, 56, current processing connection information storage area, 57, preprocessing connection information Storage area, 60 ... label link unit, 70 ... scanning mask, 71 ... target pixel, 72a to 72d ... reference pixels

Claims (7)

A labeling processing unit that attaches a label to a pixel of interest by referring to a label attached to a reference pixel adjacent to the pixel of interest of the image data;
A first labeling information storage unit that stores labeling information related to labels given by the labeling processing unit;
A second labeling information storage unit that stores the labeling information transferred from the first labeling information storage unit;
When the number of labels related to the label given by the labeling processing unit reaches the number of flash labels, the labeling information stored in the first labeling information storage unit is transferred to the second labeling information storage unit and the first labeling information is transferred. A labeling processing device, comprising: a flash control unit that flashes a storage unit.   When the first labeling information storage unit is flushed, the labeling processing unit provides preprocessing connection information indicating a connection relationship between a label given before flushing and a label given after flushing. The labeling processing device according to claim 1, wherein the labeling processing device stores the information in the information storage unit.   When the number of labels related to the label given by the labeling processing unit reaches the number of flash labels in the middle of the line of the image data, the flash control unit finishes the processing of the line by the labeling processing unit. The labeling processing apparatus according to claim 1 or 2, wherein the first labeling information storage unit is flashed.   The labeling processing device according to claim 1, wherein an access speed of the first labeling information storage unit is faster than an access speed of the second labeling information storage unit.   5. The labeling processing device according to claim 1, wherein the storage capacity of the first labeling information storage unit is smaller than the storage capacity of the second labeling information storage unit.   The number of flash labels is the storage capacity related to the labeling information corresponding to the number of flash labels and the number of labels given by the labeling processing unit becomes the number of flash labels in the middle of the line, and then the remaining portion of the line. The storage capacity related to the labeling information corresponding to the predicted value of the number of labels to be newly added, and the storage capacity are set so as not to exceed the storage capacity of the first labeling information storage unit. The labeling processing device according to claim 1.   The labeling processing unit, as the labeling processing result, generates connection information indicating a connection relationship between labels and additional information indicating the number and existence range of labeled pixels for each label, and the first labeling information. A pixel / label information indicating a correspondence between each pixel in the image data and a label given to each pixel is generated and stored in the second labeling information storage unit. The labeling processing apparatus according to any one of claims 1 to 6.

Images