JP7403722B1 - Template matching device and template matching method - Google Patents

Abstract

The pixel comparison unit (120) extracts a plurality of feature pixels from the template image, extracts a plurality of target pixels from the observed image to be compared with the feature pixel for each extracted feature pixel, and Pixel distances to the plurality of extracted feature pixels are calculated. A position detection unit (130) detects a position corresponding to the template image from the observed image based on the pixel distance of each target pixel.

Description

The present disclosure relates to speeding up template matching.

Template matching is a technique for detecting the position of a template image from an observed image.
The observed image is an image obtained by photographing using a camera or the like.
A template image is an image that includes an object to be searched.

Template matching is used, for example, as follows.
In the crime prevention and security fields, template matching is used for face recognition or object detection.
Template matching is used for product inspection on factory lines.

By speeding up the template matching process, real-time detection, for example, can be realized.
Sparse template matching has been proposed to speed up processing. Sparse template matching is a method of sparsifying the pixels being compared.

Patent Document 1 proposes a circuit configuration for speeding up template matching processing.
Specifically, a circuit configuration has been proposed that can simultaneously compare a characteristic pixel of a template image and a plurality of target pixels of an observed image.

Japanese Patent Application Publication No. 03-094387

The control in Patent Document 1 is a simple control in which specific pixels are sequentially referred to in order to extract a target pixel from an observed image.
Therefore, with the circuit configuration of Patent Document 1, it is not possible to efficiently refer to target pixels that have been made sparse in various ways by sparse template matching.
Therefore, with the circuit configuration of Patent Document 1, speeding up by sparse template matching cannot be realized.

The present disclosure aims to realize speeding up by sparse template matching.

The template matching device of the present disclosure includes:
a feature pixel extraction unit that extracts a plurality of feature pixels from the template image;
a target pixel extraction unit that extracts a plurality of target pixels from the observed image, which are compared with the characteristic pixel for each extracted feature pixel;
a distance calculation unit that calculates pixel distances to the plurality of feature pixels extracted for each extracted target pixel;
A position detection unit detects a position corresponding to the template image from the observed image based on the pixel distance of each target pixel.

According to the present disclosure, speeding up by sparse template matching can be achieved.

FIG. 1 is a configuration diagram of a template matching device 100 in Embodiment 1. FIG. 2 is a configuration diagram of a pixel comparison unit 120 in the first embodiment. 5 is a flowchart of a template matching method in Embodiment 1. 5 is a flowchart of step S110 in the first embodiment. 5 is a flowchart of step S120 in the first embodiment. FIG. 2 is a configuration diagram of a template matching device 100 in a second embodiment. FIG. 2 is a configuration diagram of a pixel comparison unit 120 in Embodiment 2. 7 is a flowchart of a template matching method in Embodiment 2. Flowchart of step S220 in Embodiment 2. 7 is a flowchart of a template matching method in Embodiment 3. Flowchart of step S320 in Embodiment 3. FIG. 1 is a hardware configuration diagram of a template matching device 100 in an embodiment.

In the embodiments and drawings, the same or corresponding elements are denoted by the same reference numerals. Descriptions of elements assigned the same reference numerals as explained elements will be omitted or simplified as appropriate. Arrows in the figure mainly indicate the flow of data or processing.

Embodiment 1.
The form of sparse template matching will be explained based on FIGS. 1 to 5.

***Explanation of configuration***

The configuration of the template matching device 100 will be explained based on FIG. 1.
The template matching device 100 includes elements such as an observed image acquisition section 110, a pixel comparison section 120, a position detection section 130, and an overall control section 170.
Furthermore, the template matching device 100 includes an element called a storage section. Specifically, the template matching device 100 includes storage units such as an observed image storage unit 140, a template image storage unit 150, and a coordinate information storage unit 160.

The components of the template matching device 100 are connected to each other via signal lines.

The configuration of the pixel comparison section 120 will be explained based on FIG. 2.
The pixel comparison section 120 includes elements such as a feature pixel extraction section 121, a target pixel extraction section 122, and a distance calculation section 123.

***Operation explanation***
The operation procedure of the template matching device 100 corresponds to a template matching method.

The template matching method will be explained based on FIG. 3.
When the template matching apparatus 100 is activated, the overall control unit 170 starts implementing the template matching method.
In the template matching method, the observed image acquisition section 110, pixel comparison section 120, position detection section 130, and various storage sections are controlled by the overall control section 170.

In step S110, the observation image acquisition unit 110 acquires an observation image.
The observed image is an image of the target object, and is obtained by photographing the target object using a photographing device such as a camera.
The target object is an object that is the target of template matching.

The procedure of step S110 will be explained based on FIG. 4.
In step S111, the observed image is input to the template matching device 100 one or more pixels in sequence. For example, one or more pixels of an observed image are input to the template matching device 100 from a camera.
The observation image acquisition unit 110 receives the input observation image.

In step S112, the observed image acquisition unit 110 stores the received one or more pixels in the observed image storage unit 140.

The observed image acquisition unit 110 also adds 1 to update the number of times the observed image is stored.
The number of times observation images are stored is managed using an internal counter. The initial value of the number of times observation images are stored is zero.

In step S113, the observed image acquisition unit 110 determines whether sufficient pixels of the observed image have been stored in the observed image storage unit 140 for implementing the template matching method.
For example, if the number of stored pixels reaches a threshold, enough pixels of the observed image are stored. The threshold value is, for example, the number of pixels of the observed image, and is determined in advance.
If not enough pixels of the observed image are stored, the process advances to step S111.
If enough pixels of the observed image have been stored, step S110 ends.

Returning to FIG. 3, the explanation will continue from step S120.
In step S120, the pixel comparison unit 120 calculates a pixel distance for each target pixel in the observed image.
The pixel distance is the distance of the target pixel to a plurality of characteristic pixels in the template image.
The template image is an image showing the target object, and is stored in the template image storage unit 150 in advance.
A feature pixel is a pixel in which a feature point is reflected, and is extracted from the template image.
A target pixel is a pixel to be compared with a feature pixel and is extracted from an observed image.

The procedure of step S120 will be explained based on FIG. 5.
In step S121, the feature pixel extraction unit 121 extracts one unextracted feature pixel from the template image.
Then, the feature pixel extraction unit 121 passes the extracted feature pixels to the distance calculation unit 123.

Feature pixels are extracted as follows.
First, the feature pixel extraction unit 121 generates the address of the feature pixel in the template image based on the coordinate information and an internal counter for the number of feature pixels.
The coordinate information indicates the coordinate values of each of the plurality of feature pixels in the template image. The coordinate information is specified by the user and stored in the coordinate information storage section 160 in advance.
Then, the feature pixel extraction unit 121 extracts feature pixels from the template image based on the generated address.

The "feature pixel" in step S122 means one feature pixel extracted in step S121.

In step S122, the target pixel extraction unit 122 extracts one unextracted target pixel from the observed image.
Then, the target pixel extraction unit 122 passes the extracted target pixel to the distance calculation unit 123.

The target pixel is extracted as follows, similar to the feature pixel.
First, the target pixel extraction unit 122 generates the address of the target pixel in the observed image based on the coordinate information.
Then, the target pixel extraction unit 122 extracts the target pixel from the observed image based on the generated address.

The "feature pixel" in step S123 means one feature pixel extracted in step S121.
The "target pixel" in step S123 means one target pixel extracted in step S122.

In step S123, the distance calculation unit 123 calculates the distance between the feature pixel and the target pixel.

For example, the distance is calculated as follows.
The distance calculation unit 123 calculates the difference (absolute value) between the pixel value of the feature pixel and the pixel value of the target pixel. The calculated difference (absolute value) is the distance between the feature pixel and the target pixel.

In step S124, the target pixel extraction unit 122 determines whether there are any unextracted target pixels.

The determination is performed as follows.
First, the target pixel extraction unit 122 adds 1 to update the number of target pixels to be processed.
The number of target pixels to be processed is managed using an internal counter. The initial value of the number of target pixels to be processed is zero.
Then, the target pixel extraction unit 122 compares the number of processed target pixels with a threshold value. The threshold value is a predetermined value designated by the user.
If the number of target pixels processed is equal to the threshold value, there are no unextracted target pixels.
If the number of target pixels to be processed is less than the threshold, there are target pixels that have not yet been extracted.

If there is an unextracted target pixel, the process advances to step S122.
If there are no unextracted target pixels, the process advances to step S125.

The "target pixel" in step S125 means each of the plurality of target pixels extracted in step S122.

In step S125, the distance calculation unit 123 updates the pixel distance for each target pixel based on the distance between the feature pixel and the target pixel. Thereby, the pixel distance of each target pixel is calculated.

For example, the pixel distance is updated as follows.
The distance calculation unit 123 adds the distance between the feature pixel and the target pixel to update the pixel distance. The initial value of pixel distance is zero.

In step S126, the feature pixel extraction unit 121 determines whether there are any feature pixels that have not been extracted.

The determination is performed as follows.
First, the feature pixel extraction unit 121 adds 1 to update the number of processed feature pixels.
The number of processed feature pixels is managed using an internal counter. The initial value of the number of processed feature pixels is zero.
Then, the feature pixel extraction unit 121 compares the number of processed feature pixels with a threshold value. The threshold value is a predetermined value designated by the user.
If the number of processed feature pixels is equal to the threshold, there are no unextracted feature pixels.
If the number of processed feature pixels is less than the threshold, there are feature pixels that have not been extracted.

If there are unextracted feature pixels, the process advances to step S121.
At this time, the feature pixel extraction unit 121 initializes the number of target pixels to be processed. Further, the feature pixel extraction unit 121 adds 1 to update the number of processed feature pixels.
The number of processed feature pixels is managed using an internal counter. The initial value of the number of processed feature pixels is zero.

If there are no unextracted feature pixels, step S120 ends.

Returning to FIG. 3, step S130 will be explained.
In step S130, the position detection unit 130 detects a position corresponding to the template image from the observed image based on the pixel distance of each target pixel.
Then, the position detection unit 130 outputs the detection result.

For example, the position corresponding to the template image is detected as follows.
The position detection unit 130 selects the position of the target pixel with the smallest pixel distance. The selected position is the position corresponding to the template pixel.

After step S130, the processing of the template matching method ends.

***Effects of Embodiment 1***
Embodiment 1 aims at speeding up position detection through efficient sparse template matching processing.
Embodiment 1 implements sparse template matching processing.
In the first embodiment, a feature pixel and a target pixel are extracted based on coordinate information (position information) and an internal counter for the number of feature pixels, and the extracted feature pixel and the extracted target image are compared. Therefore, position detection by sparse template matching processing is sped up.

***Example of Embodiment 1***
Instead of storing all the pixels of the template image, the template image storage unit 150 may store all the characteristic pixels in the template image.
The feature pixel extraction unit 121 extracts feature pixels from the template image storage unit 150 based on an internal counter of the number of feature pixels without referring to coordinate information.
For example, the target pixel extraction unit 122 extracts target pixels corresponding to the feature pixels extracted by the feature pixel extraction unit 121.

Step S120 may be started before all pixels of one observed image are read in step S110. In this case, the number of loops in step S120 is changed depending on the size of the read observation image.

Multiple template images may be used.
In this case, an internal counter for the number of template images is added, a branch process occurs according to the value of the internal counter, and position detection is performed by comparing a plurality of pixel distances to a plurality of template images.

Embodiment 2.
With regard to a mode for speeding up sparse template matching, the main differences from Embodiment 1 will be explained based on FIGS. 6 to 9.

***Explanation of configuration***
The configuration of the template matching device 100 will be explained based on FIG. 6.
The configuration of template matching device 100 differs from the configuration in Embodiment 1 in that it includes a plurality of observed image storage units 140.

The configuration of the pixel comparison section 120 will be explained based on FIG. 7.
The pixel comparison section 120 includes elements such as a feature pixel extraction section 121, a plurality of target pixel extraction sections 122, a plurality of distance calculation sections 123, and a comparison range control section 124.

The plurality of target pixel extraction units 122 are arranged in parallel and operate simultaneously.
The plurality of distance calculation units 123 are arranged in parallel and operate simultaneously.
The target pixel extraction unit 122 and the distance calculation unit 123 are connected in a one-to-one correspondence.

The comparison range control unit 124 determines two or more target pixels whose pixel distances are to be calculated from among the plurality of target pixels.

***Operation explanation***
The template matching method will be explained based on FIG. 8.
In step S210, the observation image acquisition unit 110 acquires an observation image.
The procedure of step S210 is the same as the procedure of step S110 in the first embodiment. However, the observed images are stored in each of the plurality of observed image storage units 140.

In step S220, the pixel comparison unit 120 calculates a pixel distance for each target pixel in the observed image.

The procedure of step S220 will be explained based on FIG. 9.
In step S221, the feature pixel extraction unit 121 extracts one unextracted feature pixel from the template image.
Step S221 is the same as step S121 in the first embodiment.

In step S222, the plurality of target pixel extraction units 122 simultaneously extract mutually different target pixels. Thereby, multiple target pixels are extracted at once.

A plurality of target pixels are extracted as follows.
Different offset addresses are set in each observed image storage unit 140.
Each target pixel extraction unit 122 extracts the associated observed image storage unit 140 based on the coordinate information and an address based on the internal counter of the number of feature pixels, and the offset address of the associated observed image storage unit 140. Extract the target pixel from the observed image stored in .

If the number of observation image storage units 140 is greater than the number of target pixel extraction units 122, the comparison range control unit 124 determines two or more observation image storage units 140 to be used in step S222 based on the coordinate information. .

In step S223, the plurality of distance calculation units 123 simultaneously calculate distances between the feature pixel and the target pixel for different target pixels. Thereby, the distance between the feature pixel and each target pixel is calculated at once. The calculation method is the same as the method in the first embodiment.

Specifically, each distance calculation unit 123 calculates the distance between the feature pixel and the target pixel for the target pixel extracted by the associated target pixel extraction unit 122.

Storage elements such as shift registers can be used to control the comparison range. A shift register is an example of a storage element in which data can be moved by simple shift processing.
A plurality of target pixels are stored in the shift register. A part (certain range) of the plurality of target pixels is compared with the characteristic pixels. In other words, the distance between each of some target pixels and the feature pixel is calculated.
Some of the feature pixels are extracted from the storage element by performing a shift process on the storage element by the amount of coordinate information. Some of the feature pixels have a number of digits specified in advance by the user.

In step S224, the plurality of distance calculation units 123 update pixel distances for different target pixels based on the distance between the feature pixel and the target pixel. Thereby, the pixel distance of each target pixel is calculated simultaneously. The updating method is the same as the method in the first embodiment.

Specifically, each distance calculation unit 123 updates the pixel distance of the target pixel extracted by the associated target pixel extraction unit 122.

In step S225, the feature pixel extraction unit 121 determines whether there are any unextracted feature pixels. Step S225 is the same as step S126 in the first embodiment.

If there are unextracted feature pixels, the process advances to step S221. At this time, the feature pixel extraction unit 121 adds 1 to update the number of processed feature pixels.
If there are no unextracted feature pixels, step S220 ends.

Returning to FIG. 8, step S230 will be explained.
In step S230, the position detection unit 130 detects a position corresponding to the template image from the observed image based on the pixel distance of each target pixel.
Step S230 is the same as step S130 in the first embodiment.

After step S230, the processing of the template matching method ends.

***Effects of Embodiment 2***
Embodiment 2 speeds up sparse template matching processing by processing feature pixels of a template image and a plurality of target pixels of an observed image simultaneously.
In the second embodiment, a plurality of target pixel extraction units 122 and distance calculation units 123 are arranged. Thereby, the distances between the feature pixel of the template image and each of the plurality of target pixels of the observed image can be calculated simultaneously. Therefore, sparse template matching processing is sped up.
The template matching device 100 includes a comparison range control unit 124, and selects target pixels to be compared from pixels extracted from a plurality of observed image storage units 140 by shift processing. This makes it possible to realize a flexible circuit configuration for other image processing without placing any limit on the number of divisions when storing observed images.

Embodiment 3.
With regard to the mode for speeding up sparse template matching, the main differences from Embodiment 2 will be explained based on FIGS. 10 and 11.

***Explanation of configuration***
The configuration of template matching device 100 is the same as that in the second embodiment.

***Operation explanation***
The template matching method will be explained based on FIG. 10.
In step S310, the observation image acquisition unit 110 acquires an observation image.
Step S310 is the same as step S210 in the second embodiment.

In step S320, the pixel comparison unit 120 calculates a pixel distance for each target pixel in the observed image.

The procedure of step S320 will be explained based on FIG. 11.
In step S321, the feature pixel extraction unit 121 extracts one unextracted feature pixel set from the template image.

A feature pixel set is one or more feature pixels.
Specifically, the feature pixel set is one or more pixels that are located a fixed distance apart from two or more feature pixels that are on the same line in the template image.

In step S322, the multiple target pixel extraction units 122 extract multiple target pixels from the observed image. The plurality of target pixels to be extracted are different from each other.

The number of target pixels is the same as the sum of the number of target pixels and the distance in the feature pixel set to one feature pixel.

In step S323, the plurality of distance calculation units 123 calculate the distance between each feature pixel in the feature pixel set and the target pixel for each target pixel corresponding to each feature pixel in the feature pixel set. The calculation method is the same as the method of step S223 in the second embodiment.

Some of the feature pixels are extracted from the storage element by performing a shift process on the storage element by the coordinate information of the feature pixel and the fixed distance of the feature pixel.

In step S324, the plurality of distance calculation units 123 update the pixel distance for each target pixel based on the distance between each feature pixel in the feature pixel set and the target pixel. The updating method is the same as the method in the first embodiment.

Specifically, each distance calculation unit 123 updates the pixel distance for the target pixel extracted by the associated target pixel extraction unit 122.

In step S325, the feature pixel extraction unit 121 determines whether there is any feature pixel set that has not been extracted. Step S325 is similar to step S126 in the first embodiment.

If there is a feature pixel set that has not been extracted, the process advances to step S321. At this time, the feature pixel extraction unit 121 adds 1 to update the number of processed feature pixels.
If there is no unextracted feature pixel set, step S320 ends.

Returning to FIG. 10, step S330 will be explained.
In step S330, the position detection unit 130 detects a position corresponding to the template image from the observed image based on the pixel distance of each target pixel.
Step S330 is the same as step S230 in the second embodiment.

After step S330, the processing of the template matching method ends.

***Effects of Embodiment 3***
Embodiment 3 speeds up sparse template matching processing when constraints are imposed on the coordinates of feature pixels of a template image.
According to the third embodiment, extraction of a plurality of target pixels is made more efficient. Furthermore, it is possible to reduce the number of repetitions in step S320. The greater the number of feature pixels included in the feature pixel set, the greater the effect of reducing the number of repetitions. Furthermore, the smaller the fixed distance is, the more efficiently extraction of a plurality of target pixels becomes.

＊＊＊Supplementary information regarding the implementation form＊＊＊
The hardware configuration of the template matching device 100 will be explained based on FIG. 12.
The template matching device 100 includes a processing circuit 101 and an input/output interface 102.
The processing circuit 101 is hardware that implements an observation image acquisition section 110, a pixel comparison section 120, a position detection section 130, an observation image storage section 140, a template image storage section 150, a coordinate information storage section 160, and an overall control section 170. .
The processing circuit 101 may be dedicated hardware or may be a processor that executes a program stored in memory.
The program can be recorded (stored) in a computer-readable manner on a non-volatile recording medium such as an optical disk or a flash memory.

If processing circuit 101 is dedicated hardware, processing circuit 101 is, for example, a single circuit, a composite circuit, a programmed processor, a parallel programmed processor, an ASIC, an FPGA, or a combination thereof.
ASIC is an abbreviation for Application Specific Integrated Circuit.
FPGA is an abbreviation for Field Programmable Gate Array.

The template matching device 100 may include a plurality of processing circuits that replace the processing circuit 101.

In the processing circuit 101, some functions may be realized by dedicated hardware, and the remaining functions may be realized by software or firmware.

In this way, the functions of the template matching device 100 can be realized by hardware, software, firmware, or a combination thereof.

Each embodiment is an illustration of a preferred form and is not intended to limit the technical scope of the present disclosure. Each embodiment may be implemented partially or in combination with other embodiments. The procedures described using flowcharts and the like may be modified as appropriate.

The template matching device 100 may be read as an "image processing device" or an "image processing circuit."
The “device” of the template matching device 100 may be read as a “circuit”.
The "unit" of each element of the template matching device 100 may be read as "process", "process", "circuit", or "circuitry".

Reference Signs List 100 template matching device, 101 processing circuit, 102 input/output interface, 110 observed image acquisition unit, 120 pixel comparison unit, 121 feature pixel extraction unit, 122 target pixel extraction unit, 123 distance calculation unit, 124 comparison range control unit, 130 position Detection unit, 140 Observation image storage unit, 150 Template image storage unit, 160 Coordinate information storage unit, 170 Overall control unit.

Claims (2)

a feature pixel extraction unit that extracts a plurality of feature pixel sets from the template image by using a plurality of feature pixels located between a fixed distance from each other on the same line in the template image as feature pixel sets; ,
a target pixel extraction unit that extracts a plurality of target pixels from the observed image to be compared with the characteristic pixel set for each extracted feature pixel set;
a comparison range control unit that determines two or more target pixels for which pixel distances to the plurality of characteristic pixel sets are calculated from among the plurality of target pixels for each of the plurality of extracted target pixels;
a distance calculation unit that calculates a pixel distance to the plurality of characteristic pixel sets for each determined target pixel;
a position detection unit that detects a position corresponding to the template image from the observed image based on the pixel distance of each target pixel;
A template matching device comprising: Extracting a plurality of feature pixel sets from the template image, using a plurality of feature pixels located between a fixed distance among two or more feature pixels on the same line in the template image as a feature pixel set;
Extract multiple target pixels from the observed image to be compared with the feature pixel set for each extracted feature pixel set,
determining two or more target pixels for which pixel distances to the plurality of feature pixel sets are calculated from among the plurality of target pixels for each of the plurality of extracted target pixels;
Calculating pixel distances to the plurality of characteristic pixel sets for each determined target pixel,
A template matching method that detects a position corresponding to the template image from the observed image based on the pixel distance of each target pixel.

Images