JP7188426B2 - HARDWARE ACCELERATOR, IMAGE PROCESSING DEVICE AND IMAGE PROCESSING METHOD - Google Patents

Description

The present invention relates to a hardware accelerator, an image processing device and an image processing method.

LBPs (Local Binary Patterns), in which local expressions of images are used as feature quantities, are used for image recognition, classification, and the like. The LBP has the characteristics of being less susceptible to changes in illumination and being able to be calculated at high speed.

In the feature point detection described in Patent Document 1, part or all of the processing is performed by dedicated hardware (hereinafter, dedicated hardware is referred to as a "hardware accelerator") in order to speed up the processing. It is also expected to be realized. By the way, LBP includes derived methods such as Uniform LBP and Improved LBP, and it is preferable to select an appropriate method according to the application.

JP 2014-199505 A

However, if each method of derived LBP is individually implemented by a hardware accelerator as it is, the circuit scale of the hardware accelerator will increase.

Therefore, the present invention has been made to solve the above problems, and an object of the present invention is to provide a hardware accelerator, an image processing apparatus, and an image processing method suitable for supporting a plurality of LBPs.

In order to achieve the above object, the hardware accelerator of the present invention is
a reference value derivation unit having a plurality of types of filters for deriving a reference value by using a filter for the pixel value of each pixel and the pixel values of a plurality of pixels surrounding each pixel of the image; ,
Binary pattern generation with multiple comparison modes, wherein a binary pattern is generated by comparing the pixel value of each pixel of the image or the reference value with the pixel values of a plurality of pixels surrounding each pixel. Department and
a binary pattern conversion unit that converts the binary pattern into a post-conversion binary pattern;
A reference value derived by using an arbitrarily set filter among the plurality of types of filters in a comparison mode arbitrarily set among the plurality of types of comparison modes is applied to a plurality of surrounding pixels of the respective pixels. A histogram generation unit that generates a histogram based on the binary pattern generated by comparing the pixel value of the pixel or the post-conversion binary pattern,
At least one of the reference value derivation unit, the binary pattern conversion unit, and the histogram generation unit according to processing corresponding to generation of LBP (Local Binary Pattern) selected by a user via an input device ; and outputting the binary pattern obtained by selectively using the binary pattern generation unit, the converted binary pattern, or the histogram.

According to the present invention, it is possible to provide a hardware accelerator, an image processing apparatus, and an image processing method suitable for supporting a plurality of LBPs.

FIG. 4 is an explanatory diagram of four processes executed by the hardware accelerator of the image processing apparatus according to the embodiment of the present invention; FIG. 5 is a diagram illustrating combinations of each LBP and processing to be executed in the LBP setting processing of the image processing apparatus according to the embodiment of the present invention; 1 is a block diagram of an image processing device according to an embodiment of the present invention; FIG. 6 is a flowchart of LBP setting processing of the image processing apparatus according to the embodiment of the present invention; FIG. 4 is an explanatory diagram of various LBP feature amount generation processing of the image processing apparatus according to the embodiment of the present invention; 4 is a flow chart of image processing of the image processing apparatus according to the embodiment of the present invention; 4 is a flowchart of various LBP feature amount generation processing of the image processing apparatus according to the embodiment of the present invention; 5 is a flowchart of reference value calculation processing of the image processing apparatus according to the embodiment of the present invention; 4 is a flowchart of binary pattern generation processing of the image processing apparatus according to the embodiment of the present invention; 4 is a flowchart of binary pattern conversion processing of the image processing apparatus according to the embodiment of the present invention; 4 is a flowchart of histogram generation processing of the image processing apparatus according to the embodiment of the present invention;

An image processing apparatus having a hardware accelerator according to embodiments of the present invention will be described in detail below with reference to the drawings.

An image processing apparatus equipped with a hardware accelerator according to an embodiment of the present invention combines the four processes shown in FIG. 1 to convert an input image into normal LBP, improved LBP, CS (Center-Symmetric) LBP, RI (Rotation Invariant) LBP, Uniform LBP, LBP Histogram, Improved RI LBP Histogram.

(Configuration of image processing device)
As shown in FIG. 3, the image processing apparatus 10 includes a control section 20, a feature point detector 30, a storage section 40, an imaging section 50, a communication section 60, a display section 70, and an input section 80.

The control unit 20 is composed of a CPU (Central Processing Unit) or the like, and by executing a program or the like stored in the storage unit 40, each unit described later (processing selection unit 21, image acquisition unit 22, feature amount transmission unit 23 ) functions. The control unit 20 also has a clock (not shown), and can acquire the current date and time, count the elapsed time, and the like.

The feature point detector 30, which is a component of the hardware accelerator A, is for executing various processes for calculating feature amounts from the image V instead of the control unit 20. FIG. The feature point detector 30 includes a CPU and the like, and is configured by an integrated circuit (IC: Integrated Circuit) in which electronic components such as semiconductors are integrated. (pixel value acquisition unit 31, reference value calculation unit 32, binary pattern generation unit 33, binary pattern conversion unit 34, histogram generation unit 35).

A feature point detector storage unit 36, which is a component of the hardware accelerator A, is configured by a RAM (Random Access Memory) or the like, and functionally has a histogram temporary storage unit 37. FIG. The histogram temporary storage unit 37 is used for counting up in histogram generation processing, which will be described later.

The storage unit 40 is composed of ROM (Read Only Memory), RAM, etc., and part or all of the ROM is composed of electrically rewritable memory (flash memory, etc.). The storage unit 40 functionally includes a captured image storage unit 41 , a reference value storage unit 42 , a binary pattern storage unit 43 , a histogram storage unit 44 and various setting storage unit 45 . The ROM stores programs executed by the control unit 20 and the feature point detector 30 and data necessary for executing the programs. RAM stores data that is created or changed during program execution.

In this embodiment, the captured image storage unit 41 stores an image V captured by the imaging unit 50 . In the present embodiment, one imaging device 51 to be described later is used for simplification of explanation. The captured image can be stored.

The reference value storage unit 42 stores a reference value calculated in a reference value calculation process, which will be described later.

The binary pattern storage unit 43 stores the comparison mode selected by the processing selection unit 21 in the LBP setting process described later and the binary pattern BP generated in the binary pattern generation process described later. Also, a binary pattern BPL converted by a binary pattern conversion process, which will be described later, based on the binary pattern BP is stored.

The histogram storage unit 44 stores histograms generated by histogram generation processing, which will be described later.

The various setting storage unit 45 stores a comparison pattern, a filter coefficient (filter), a lookup table (LUT), a correspondence relationship between each LBP and four processes as shown in FIG. , various settings such as the imaging range L in which the image V is imaged are stored in advance or by the user's input or instruction via the input unit 80 or the like.

The imaging unit 50 has an imaging device 51 that captures an image V and a driving device 52 that controls the attitude of the imaging device 51 .

The imaging device 51 includes a CMOS (Complementary Metal Oxide Semiconductor) camera in this embodiment. The imaging device 51 captures an imaging range at a preset frame rate or a user-set frame rate to generate an image V. FIG. Note that the frame rate can be changed manually or automatically by the user according to the assumed moving speed of the object to be imaged.

The driving device 52 adjusts the imaging range L by moving the position and direction of the imaging device 51 according to instructions from the input device 81 or the like, which will be described later.

The communication unit 60 has a communication device 61 which is a module for communicating with an external device or the like. The communication device 61 is a wireless module including an antenna when communicating with an external device. For example, the communication device 61 is a wireless module for short-range wireless communication based on Bluetooth (registered trademark). By using the communication unit 60, the image processing apparatus 10 can exchange data such as the image V and the feature amount with an external device or the like.

The display unit 70 includes a display device 71 configured by a liquid crystal display panel (LCD: Liquid Crystal Display).

As the display device 71, a thin film transistor (TFT), a liquid crystal, an organic EL (Electro Luminescence), or the like can be used. The display device 71 displays an image V, a feature amount list, and the like.

The input unit 80 is a resistive touch panel (input device 81 ) provided close to the display unit 70 or integrated with the display unit 70 . The touch panel may be of an infrared operation system, a projective capacitive system, or the like, and the input unit 80 may be a keyboard, a mouse, or the like instead of the touch panel. The user can select the LBP to be used by manual operation via the input unit 80 or select the LBP to be used using the display unit 70 .

Next, functional configurations of the control unit 20 and the feature point detector 30 of the image processing apparatus 10 will be described.

The control unit 20 realizes the functions of the processing selection unit 21, the image acquisition unit 22, and the feature amount transmission unit 23, and performs image processing, LBP feature amount generation processing, etc., which will be described later, together with the feature point detector 30, which is the hardware accelerator A. conduct.

The processing selection unit 21 sets the processing corresponding to the LBP selected by the user (see FIGS. 2 and 4) and sets the accelerator A. FIG. Details will be described later in the LBP setting process.

The image acquisition unit 22 acquires an image V captured by the imaging unit 50 under exposure conditions set in advance in the image processing apparatus 10 or set by the user, and stores the image V in the captured image storage unit 41 .

The feature amount transmission unit 23 transmits the binary pattern BP, BPL, or histogram to an external device or the like according to the LBP selected by the user.

The feature point detector 30 realizes the functions of a pixel value acquisition unit 31, a reference value calculation unit 32, a binary pattern generation unit 33, a binary pattern conversion unit 34, and a histogram generation unit 35, and together with the control unit 20 performs image processing, LBP feature volume generation processing, etc.

The pixel value acquisition unit 31 acquires the pixel value of each pixel and the pixel values of eight pixels surrounding the pixel from the image V stored in the captured image storage unit 41 . The pixel value acquisition unit 31 transmits the acquired image V to the reference value calculation unit 32 or the binary pattern generation unit 33 according to the LBP selected by the user.

The reference value calculation unit 32 uses a filter to obtain a reference value for the pixel values of each pixel and eight pixels surrounding each pixel of the image V transmitted from the pixel value acquisition unit 31. calculate. As shown in FIG. 1, the filter has (1) a central pixel value of 3×3 pixel values as a reference value (corresponding to the first filter from the top of the reference value calculation column in FIG. 1) ( Normal LBP, RI LBP, Uniform LBP), (2) The average value of 3 × 3 pixel values is the reference value (the filter coefficient is 1 for both the center and the periphery (the second filter from the top of the reference value calculation column in FIG. 1) (Equivalent to)) (Improved LBP, Improved RI LBP Histogram), (3) The pixel value in the center of the 3 × 3 pixel value is given the maximum weight, the upper, lower, left, and right sides of the center are weighted by half, and the four corners are further halved. (corresponding to the third filter from the top of the reference value calculation column in FIG. 1). Note that the user can arbitrarily change or add the numerical value of the filter.

The binary pattern generation unit 33 generates a binary pattern BP by comparing the pixel value of each pixel in the captured image or the reference value with the pixel values of eight pixels surrounding each pixel. . As for the comparison method, as shown in FIG. 5, (1) a reference value comparison mode for comparing the reference value and the surrounding 8 pixels, and (2) the central pixel for point-symmetrical comparison of the surrounding 8 pixels. There is a pixel comparison mode, and (3) an adjacent pixel comparison mode in which adjacent pixels are compared. When Normal LBP, Improved LBP, RI LBP, Uniform LBP, LBP Histogram, and Improved RI LBP Histogram are selected, mode (1) is used for comparison, and when CS LBP is selected, mode (2) is used for comparison. The mode can also be arbitrarily changed by the user. As shown in FIG. 1, 0 is set when the comparison source is less than the comparison destination, and 1 is set otherwise.

The binary pattern conversion unit 34 converts the binary pattern generated by the binary pattern generation unit 33 into a post-conversion binary pattern using a pattern conversion table, which is an 8-bit lookup table (LUT), as shown in FIGS. Convert to BPL. For example, for RI LBP (and Improved RI LBP Histogram), the following 8-bit conversion is performed.
00000001(1)→00000001(1)
00000010 (2) → 00000001 (1)
00000100 (4) → 00000001 (1)
00001000 (8) → 00000001 (1)
00010000 (16) → 00000001 (1)
00100000 (32) → 00000001 (1)
01000000 (64) → 00000001 (1)
10000000 (128) → 00000001 (1)

00000011(3)→00000011(3)
00000110 (6) → 00000011 (3)
00001100 (12) → 00000011 (3)
00011000 (24) → 00000011 (3)
00110000 (48) → 00000011 (3)
01100000 (96)→00000011 (3)
11000000 (192)→00000011 (3)
10000001 (129)→00000011 (3)

…

In other words, the following table corresponds to the bit pattern rotation by bit shift.
tb[1]=1
tb[2]=1
tb[3]=3
tb[4]=1
tb[6]=3
tb[8]=1
tb[12]=3
tb[16]=1
tb[24]=3
tb[32]=1
tb[48]=3
tb[64]=1
tb[96]=3
tb[128]=1
tb[129]=3
tb[192]=3
…

On the other hand, in Uniform LBP, for example, bit patterns are classified into spot, flat (spot/flat), line end, edge, and corner, and the following pattern conversion table is used. to use. Please refer to the web page "Theory and Implementation of Local Binary Pattern".
00000000(1)→00000001(1)=spot
11111111(255)→00000002(2)=spot/flat
00000110(6)→00000004(4)=line end
00001100(12)→00000004(4)=line end
00011000(24)→00000004(4)=line end
…
11000011 (195)→00000008 (8)=edge
10000111(135)→00000008(8)=edge
00001111(15)→00000008(8)=edge
…
00000111(7)→00000010(16)=corner
00001110(14)→00000010(16)=corner
00011100 (28)→00000010 (16)=corner
…

A histogram generator (frequency distribution calculator) 35 generates a histogram based on the binary pattern BP or the converted binary pattern BPL. In the case of the LBP Histogram and the Improved RI LBP Histogram, the histogram generator 35 counts up the bin corresponding to the binary pattern and counts up all the pixels to generate the histogram. After calculating a histogram for each block of a set size (pixel block of 64 pix×64 pix in this embodiment), a histogram is generated by concatenating the histograms. A histogram of the entire image V may be generated.

The functional configurations of the control unit 20 and the feature point detector 30 have been described above. 3 to 11, normal LBP, improved LBP, CS LBP, RI LBP, uniform shown in FIG. Image processing, various LBP feature amount generation processing, and the like executed to generate feature amounts of various LBPs such as LBP, LBP Histogram, and Improved RI LBP Histogram will be specifically described.

(LBP setting process)
First, before image processing is performed, LBP setting processing (see FIGS. 4 and 2) is performed to configure the processing to be executed by the hardware accelerator A according to the LBP selected by the user, as shown in FIG.

First, the user selects an LBP for generating feature amounts of pixels of the image V. FIG. The processing selection unit 21 refers to the combination of each LBP and the processing to be executed stored in the various setting storage unit 45 (see FIG. 2), and confirms the processing corresponding to the LBP selected by the user (step S101).

If the reference value calculation process corresponds to the LBP selected by the user (step S102; Yes), the reference value calculation unit 32 is made executable (step S103). If the reference value calculation process does not correspond to the LBP selected by the user (step S102; No), the reference value calculation unit 32 is not enabled.

Proceeding to step S<b>104 , the processing selection unit 21 selects a comparison pattern corresponding to the LBP selected by the user from the various setting storage unit 45 and stores it in the binary pattern storage unit 43 .

Proceeding to step S105, when the binary pattern conversion process corresponds to the LBP selected by the user (step S105; Yes), the binary pattern conversion unit 34 is made executable (step S106). If the binary pattern conversion process does not correspond to the LBP selected by the user (step S105; No), the binary pattern conversion unit 34 is not enabled.

Proceeding to step S107, if the histogram generation process corresponds to the LBP selected by the user (step S107; Yes), the histogram generation unit 35 is enabled (step S108). If the histogram generation process does not correspond to the LBP selected by the user (step S107; No), the histogram generation unit 35 is not enabled. With this, the LBP setting process is completed.

(Image processing)
After the LBP setting process is completed, as shown in FIG. 6, image processing for calculating the LBP feature amount is executed for each pixel of the imaged image V. FIG. Assume that the total number of pixels of the image V is k.

First, the imaging unit 50 captures an image of a designated imaging range set at a set frame rate, and the image acquiring unit 22 stores the captured image V in the captured image storage unit 41 (step S201).

Next, the acquisition of the pixel value of the n-th (1≤n≤k) pixel is started (step S202). The pixel value acquisition unit 31 acquires the pixel value of the n-th pixel and the pixel values of eight pixels surrounding the n-th pixel from the image V stored in the captured image storage unit 41 (step S203).

Proceeding to the next step, various LBP feature amount generation processes (described later) corresponding to the LBP set by the user are executed (step S204).

Proceeding to step 205, it is determined whether or not n=kth pixel. If n is not the k-th pixel (step S205; No), n=n+1 is set (step S206), and the process returns to step S203. If n=kth pixel (step S205; Yes), the process proceeds to step S207.

In step S<b>207 , the feature amount transmission unit 23 sends the feature amount generated by the last executed process among the binary pattern generation process, the binary pattern conversion process, and the histogram generation process to the binary pattern storage unit 43 or the histogram storage unit 44 . , and transmitted to an external device or the like via the communication unit 60, and the image processing ends.

Subsequently, various LBP feature amount generation processing executed in step S204 of image processing will be described with reference to FIG.

(Various LBP feature amount generation processing)
First, when the reference value calculation unit 32 is in the executable state in the LBP setting process (step S301; Yes), the reference value calculation process, which will be described later, is executed (step S302), and after the execution, the process proceeds to step S303. When the reference value calculation unit 32 is not in the executable state in the LBP setting process (step S301; No), the process proceeds to step S303.

Next, binary pattern generation processing, which will be described later, is executed (step S303).

After that, the process proceeds to step S304, and when the binary pattern conversion unit 34 is in an executable state in the LBP setting process (step S304; Yes), the binary pattern conversion process described later is executed (step S305), and after execution, the process proceeds to step S306. . If the binary pattern conversion unit 34 is not executable in the LBP setting process (step S304; No), the process proceeds to step S306.

When the histogram generation unit 35 is ready for execution in the LBP setting process (step S306; Yes), the histogram generation process described later is executed (step S307), and various LBP feature amount generation processes are terminated after the execution. When the histogram generation unit 35 is not in an executable state in the LBP setting process (step S306; No), various LBP feature amount generation processes are terminated.

Subsequently, reference value calculation processing, binary pattern generation processing, binary pattern conversion processing, and histogram generation processing executed in various LBP feature amount generation processing will be described in order.

(Reference value calculation processing)
In the reference value calculation process, first, as shown in FIG. 8, the reference value calculation unit 32 acquires the filter coefficient (filter) corresponding to the LBP selected by the user from the various setting storage unit 45 (step S401).

Next, the reference value calculator 32 uses the acquired filter coefficient to perform filtering on the pixel values of the n-th pixel and eight pixels surrounding the n-th pixel (step S402).

Finally, the reference value calculation unit 32 acquires the reference value as a result of the filtering process, stores the reference value in the reference value storage unit 42 (step S403), and ends the reference value calculation process.

(Binary pattern generation processing)
Subsequently, the binary pattern generation processing will be described below with reference to FIG.

First, the binary pattern generation unit 33 acquires the comparison mode selected by the processing selection unit 21 in step S104 of the LBP setting process from the binary pattern storage unit 43 (step S501).

Next, the binary pattern generator 33 acquires the pixel value R of the comparison source. See FIG. 1 for the comparison source and comparison target pixel values (step S502).

Subsequently, the binary pattern generator 33 acquires the pixel value C to be compared. (Step S503).

After that, the binary pattern generation unit 33 compares the comparison source pixel value R and the comparison source pixel value C (step S504). When the comparison source pixel value C is larger than the comparison source pixel value R (step S504; Yes), the binary pattern generation unit 33 outputs 0 and stores it in the binary pattern storage unit 43 (step S505). If the comparison source pixel value R is equal to the comparison source pixel value C, or if the comparison source pixel value C is smaller than the comparison source pixel value R (step S504; No), the binary pattern generation unit 33 is output and stored in the binary pattern storage unit 43 (step S506).

Proceeding to step S507, it is checked whether or not n=k. If n=k (step S507; Yes), the binary pattern generation unit 33 generates the binary pattern BP (step S508), and ends the binary pattern generation process. If not n=k (step S507; No), the binary pattern generation process ends.

(Binary pattern conversion processing)
Next, the binary pattern conversion process will be described below with reference to FIG.

First, the binary pattern conversion unit 34 refers to the lookup table corresponding to the LBP selected by the user from the various setting storage unit 45, and applies it to the binary pattern BP (step S601).

The binary pattern conversion unit 34 stores the post-conversion binary pattern BPL converted from the binary pattern BP by the lookup table in the binary pattern storage unit 43 (step S602), and terminates the binary pattern conversion process.

(Histogram generation processing)
Subsequently, the histogram generation processing will be described below with reference to FIG. 11 .

First, the histogram generator 35 counts up the binary pattern BP or post-conversion binary pattern BPL in the histogram temporary storage 37 according to the user-selected LBP (step S701). At this time, the image V is divided into blocks of a size such as 3×3 pixels, the LBP is calculated for the pixels in each block, and the histogram of the LBP is generated with 256 bins for each block. Concatenate the histograms of to generate a histogram. Alternatively, a histogram of the entire image V is generated.

Subsequently, when n=k (completed count-up of all pixels) (step S702; Yes), the histogram generation unit 35 outputs the histogram generated by the count-up and stores it in the histogram storage unit 44 (step S703). , the histogram generation process ends. If n is not k (counting up of all pixels is not completed) (step S702; No), the histogram generation process ends.

As described above, the image processing apparatus 10 including the hardware accelerator A according to the present embodiment performs four processes (reference value calculation process, binary pattern generation process, binary pattern conversion process) in the hardware accelerator A according to the LBP selected by the user. , histogram generation processing) are combined in this order, and at least one of binary pattern generation processing, reference value calculation processing, binary pattern conversion processing, and histogram generation processing is executed to generate various LBP feature amounts. can. Therefore, compared to the case where hardware accelerators are configured for each of the LBPs, redundant processing can be grouped together, so it is possible to suppress an increase in the circuit scale of the hardware accelerator A, which is preferable.

[Modification]
In the above embodiment, the hardware accelerator A is used as the feature point detector 30 of the image processing apparatus 10. The hardware accelerator includes, for example, a reference value calculator, a binary pattern generator, and a binary pattern converter. A binary pattern obtained by using at least one of a calculation unit and a binary pattern conversion unit and a binary pattern generation unit or a post-conversion binary pattern is transmitted (output) to an image processing device. , Improved LBP, CS (Center-Symmetric) LBP, RI (Rotation Invariant) LBP, and Uniform LBP. Alternatively, the hardware accelerator may include a reference value calculator and a binary pattern generator, and transmit a binary pattern obtained by using at least the binary pattern generator to the image processing apparatus. Alternatively, other combinations may be used.

Alternatively, the image processing apparatus may include a feature point detector storage unit having each function of the accelerator storage unit 40 in the hardware accelerator. Alternatively, an image processing apparatus having a configuration in which two or more hardware accelerators A are mounted may be configured to be able to calculate feature amounts at a higher speed.

Alternatively, by using filters, comparison modes, lookup tables and histograms other than the above, the feature amount by LBP other than Normal LBP, Improved LBP, CS LBP, RI LBP, Uniform LBP, LBP Histogram, Improved RI LBP Histogram can also be generated.

Moreover, although the filter in the above embodiment is 3×3, it is not limited to this. 5×5 or other numbers of filters are possible.

The filters in the above embodiments include a low pass filter (LPF), a high pass filter (HPF), a band pass filter (BP), and a band elimination filter (BEF). ), etc. may be used. Alternatively, an ε filter or the like may be used. Also, the number of taps can be set to 5×5 or the like.

A clustering table such as BoF (Bag of Features) may be used as the lookup table in the above embodiment.

Although the preferred embodiments of the present invention have been described above, the present invention is not limited to such specific embodiments, and the present invention includes the invention described in the claims and their equivalents. be The invention described in the original claims of the present application is appended below.

(Appendix)
(Appendix 1)
a reference value calculation unit that calculates a reference value by using a filter for pixel values of each pixel and a plurality of pixels surrounding each pixel of the captured image;
a binary pattern generation unit that generates a binary pattern by comparing the pixel value of each pixel or the reference value of the captured image with the pixel values of a plurality of pixels surrounding each pixel;
a binary pattern conversion unit that converts the binary pattern into a post-conversion binary pattern;
a histogram generation unit that generates a histogram based on the binary pattern or the post-conversion binary pattern,
The binary pattern obtained by using at least one of the reference value calculator, the binary pattern converter, and the histogram generator, and the binary pattern generator, the converted binary pattern, or the output a histogram,
hardware accelerator.

(Appendix 2)
The reference value calculation unit outputs the pixel value of each pixel by the filter, or an average pixel value of the pixel value of each pixel and the pixel values of a plurality of pixels surrounding each pixel, or outputs the weighted average pixel value,
A hardware accelerator as described in Appendix 1.

(Appendix 3)
The binary pattern generation unit compares the pixel values of adjacent pixels, compares the pixel values of the pixels with point symmetry, or compares the pixel values of a plurality of pixels surrounding each of the pixels with the reference value. compare with
The hardware accelerator according to appendix 1 or 2.

(Appendix 4)
The binary pattern conversion unit converts with an 8-bit lookup table,
4. The hardware accelerator according to any one of appendices 1-3.

(Appendix 5)
The histogram generation unit generates a histogram by counting up bins corresponding to the binary pattern.
5. The hardware accelerator according to any one of appendices 1 to 4.

(Appendix 6)
a hardware accelerator according to any one of Appendices 1 to 6;
a control unit that acquires the binary pattern, the post-conversion binary pattern, or the histogram;
a storage unit that stores the binary pattern, the converted binary pattern, or the histogram;
comprising
Image processing device.

(Appendix 7)
hardware accelerator,
calculating a reference value by using a filter for pixel values of each pixel and a plurality of pixels surrounding each pixel of the captured image;
generating a binary pattern by comparing a pixel value of each pixel or the reference value of the captured image with pixel values of a plurality of pixels surrounding each pixel;
converting the binary pattern into a post-conversion binary pattern;
generating a histogram based on the binary pattern or the converted binary pattern;
outputting at least one of the reference value, the converted binary pattern, and the histogram, and the binary pattern;
Image processing method.

DESCRIPTION OF SYMBOLS 10... Image processing apparatus 20... Control part 21... Processing selection part 22... Image acquisition part 23... Feature-value transmission part 30... Feature point detector 31... Pixel value acquisition part 32... Reference value calculation part , 33 Binary pattern generation unit 34 Binary pattern conversion unit 35 Histogram generation unit 36 Feature point detector storage unit 37 Histogram temporary storage unit 40 Storage unit 41 Captured image storage unit 42 Reference value storage unit 43 Binary pattern storage unit 44 Histogram storage unit 45 Various setting storage unit 50 Imaging unit 51 Imaging device 52 Driving device 60 Communication unit 61 Communication device , 70... Display unit, 71... Display device, 80... Input unit, 81... Input device, A... Hardware accelerator, BP... Binary pattern, BPL... Binary pattern after conversion, L... Imaging range, V... Image

Claims (9)

a reference value derivation unit having a plurality of types of filters for deriving a reference value by using a filter for the pixel value of each pixel and the pixel values of a plurality of pixels surrounding each pixel of the image; ,
Binary pattern generation with multiple comparison modes, wherein a binary pattern is generated by comparing the pixel value of each pixel of the image or the reference value with the pixel values of a plurality of pixels surrounding each pixel. Department and
a binary pattern conversion unit that converts the binary pattern into a post-conversion binary pattern;
A reference value derived by using an arbitrarily set filter among the plurality of types of filters in a comparison mode arbitrarily set among the plurality of types of comparison modes is applied to a plurality of surrounding pixels of the respective pixels. A histogram generation unit that generates a histogram based on the binary pattern generated by comparing the pixel value of the pixel or the post-conversion binary pattern,
At least one of the reference value derivation unit, the binary pattern conversion unit, and the histogram generation unit according to processing corresponding to generation of LBP (Local Binary Pattern) selected by a user via an input device ; Outputting the binary pattern, the converted binary pattern or the histogram obtained by selectively using the binary pattern generator,
hardware accelerator. The reference value derivation unit outputs the pixel value of each pixel by the filter, or an average pixel value of the pixel value of each pixel and the pixel values of a plurality of pixels surrounding each pixel, or outputs the weighted average pixel value,
The hardware accelerator according to claim 1. The binary pattern generation unit compares the pixel values of adjacent pixels, compares the pixel values of the pixels with point symmetry, or compares the pixel values of a plurality of pixels surrounding each of the pixels with the reference value. compare with
3. The hardware accelerator according to claim 1 or 2. The binary pattern conversion unit converts with an 8-bit lookup table,
A hardware accelerator according to any one of claims 1 to 3. The histogram generation unit generates a histogram by counting up bins corresponding to the binary pattern.
A hardware accelerator according to any one of claims 1 to 4. Based on an external selection operation, the filter used by the reference value derivation unit is set from a plurality of types of filters,
A hardware accelerator according to any one of claims 1 to 5. A comparison mode to be executed by the binary pattern generation unit is set from a plurality of types of comparison modes based on an external selection operation,
A hardware accelerator according to any one of claims 1 to 6. a hardware accelerator according to any one of claims 1 to 7;
an input unit having an input device that receives selection of LBP (Local Binary Pattern) by the user;
a process selection unit that selects a process corresponding to the generation of the LBP received by the input unit;
a control unit that acquires the binary pattern, the post-conversion binary pattern, or the histogram according to the processing selected by the processing selection unit ;
a storage unit that stores the binary pattern, the post-conversion binary pattern, or the histogram acquired by the control unit ;
comprising
Image processing device. a reference value derivation unit having a plurality of types of filters for deriving a reference value by using a filter for the pixel value of each pixel and the pixel values of a plurality of pixels surrounding each pixel of the image; ,
Binary pattern generation with multiple comparison modes, wherein a binary pattern is generated by comparing the pixel value of each pixel of the image or the reference value with the pixel values of a plurality of pixels surrounding each pixel. Department and
An image processing method for a hardware accelerator, comprising: a binary pattern conversion unit that converts the binary pattern into a post-conversion binary pattern,
A reference value derived by using an arbitrarily set filter among the plurality of types of filters in a comparison mode arbitrarily set among the plurality of types of comparison modes is applied to a plurality of surrounding pixels of the respective pixels. generating a histogram based on the binary pattern generated by comparing pixel values of pixels or the transformed binary pattern;
At least one of the reference value derivation unit, the binary pattern conversion unit, and the generation step, according to a process corresponding to generation of an LBP (Local Binary Pattern) selected by a user via an input device; an output step of outputting the binary pattern, the converted binary pattern or the histogram obtained by selectively using a binary pattern generator ;
An image processing method including

Images