TWI765632B - Image recognition system based on edge detection - Google Patents

Abstract

An image recognition system comprises a camera device, a capture module, an image processing module, a facial database, a financial product database, and a computing module. The capture module captures a sub-image comprising one of the five senses of a user from the image acquired by the camera device and calculates a measurable information accordingly. The measurable information comprises a straight line or a region composed of a plurality of feature points. The facial database stores attributes and evaluation values of the measurable information. The financial product database stores a plurality of financial product information and age intervals according to a risk ratio interval. The computing module estimates an age of the user based on the measurable information, obtains the evaluation value from the facial database according to the attribute corresponding to the measurable information, and calculates the risk value accordingly, and obtains a financial product from the financial product database according to the risk value and the age.

Description

Image recognition system based on edge detection

The present invention relates to an image recognition system, in particular to an image recognition system that obtains facial feature information from images.

For example, banks, post offices and other financial institutions have set up multiple surveillance cameras in their business halls to monitor the people moving in and out of the business halls.

However, the surveillance images captured by these surveillance cameras are often used to review special events (such as masked robberies, driver theft, etc.) that occurred at a specific point in time after the fact. That is to say, the surveillance system set up by financial institutions lacks a mechanism for real-time processing and reporting based on current images. Although the surveillance system runs for a long time, it is only used as a safety device that may deter the intentional people. Furthermore, the overhead surveillance footage from a height must include all kinds of people and objects, such as waiting people, people at the counter, counter clerks, hall floors, waiting seats, ATMs, refilling machines, etc.; It is not easy to arrange a special person to watch the surveillance video and quickly locate the face of a specific person from a complex picture, not to mention that human beings are prone to miss important information in the picture due to fatigue or distraction.

In addition, for frequent high-net-worth clients, financial institutions provide exclusive financial services for clients of this segment, and allow high-net-worth clients to increase their wealth through customized commodity investment portfolios. On the other hand, financial institutions usually assist clients who are willing to carry out wealth management to make medium and long-term asset planning through their extensive marketing bases, and conduct risk assessment by understanding the current cash flow of clients and their future capital utilization plans, so as to propose a complete financial plan. Product Recommendation Program.

However, for customers who simply come to the business hall to handle general financial business, the staff or wealth management specialists of financial institutions cannot know whether the customer is a potential customer with wealth management needs based on the characteristics of the five senses. Therefore, the staff is promoting financial products. It is often a formality, unable to provide financial products that may meet the needs of this type of customers, and can only blindly sell projects that customers are not necessarily interested in, wasting the time and energy of both parties.

In view of this, the present invention proposes an image recognition system, which can obtain a face image from an image captured by a surveillance camera, and further analyze the measurable information in the facial features, thereby improving the sales success rate of financial products and reducing customer factors. Chances of bad mood due to being recommended inappropriate products.

An image recognition system according to an embodiment of the present invention includes a camera device, a capture module, an image processing module, a face database, a financial product database, and a computing module. The camera device can obtain images. The capturing module selects a block from the image and obtains a face image of a user in the block. The image processing module is electrically connected to the capture module. The image processing module captures the sub-image according to the face image and calculates measurable information according to the sub-image. The sub-image includes one of the user's facial features and surrounding skin. The measurable information includes a straight line or an area formed by a plurality of feature points. The face database includes a plurality of first table spaces, each of the first table spaces corresponds to one of the facial features and includes a plurality of first fields. Each first field stores attributes and evaluation values of measurable information. Attributes include the length of the line, the shape and color of the area. The financial product database includes a plurality of second table spaces. The second table spaces respectively correspond to a plurality of risk ratio intervals and include a plurality of second fields. Each second field stores a financial product and an age range. The computing module communicates with the image processing module, the face database and the financial product database. The computing module estimates the age of the user according to the measurable information, obtains the assessment value from the face database according to the attribute corresponding to the measurable information, calculates the risk value according to the assessment value, and calculates the risk value according to the corresponding risk ratio intervals of the risk value. One selects a second table space from the financial product database, selects a second field from the selected second table space according to the age range corresponding to the age, and outputs it.

With the above structure, the image recognition system disclosed in this case can quickly lock the face image in a specific area from the complex image captured by the surveillance camera, and analyze the age and facial features of the face image through face recognition. Measurable information, and then converts a plurality of measurable information into an evaluation score. Therefore, when a customer visits a specific counter in a business hall of a financial institution, the image recognition system disclosed in this case can further recommend suitable financial products based on the range of evaluation scores.

The above description of the present disclosure and the following description of the embodiments are used to demonstrate and explain the spirit and principle of the present invention, and provide further explanation of the scope of the patent application of the present invention.

The detailed features and advantages of the present invention are described in detail below in the embodiments, and the content is sufficient to enable any person skilled in the relevant art to understand the technical content of the present invention and implement it accordingly, and according to the content disclosed in this specification, the scope of the patent application and the drawings , any person skilled in the related art can easily understand the related objects and advantages of the present invention. The following examples further illustrate the viewpoints of the present invention in detail, but do not limit the scope of the present invention in any viewpoint.

Please refer to FIG. 1 , which is a block diagram of an image recognition system 10 according to an embodiment of the present invention. The image recognition system 10 described above is suitable for being installed in the business hall of a financial institution. The image recognition system 10 includes a camera device 1 , a capture module 2 , an image processing module 3 , a face database 4 , a financial product database 5 and an operation module 6 .

The imaging device 1 can acquire images. Please refer to FIG. 2 , which is a schematic top view of the camera device 1 installed in the business hall of a financial institution. As shown in FIG. 2 , the installation position of the camera device 1 can capture the surrounding environment of a specific counter in the business hall of the financial institution. Please refer to FIG. 3 , which shows an image captured by the camera device 1 disposed in FIG. 2 , and the image captured by the camera device 1 includes users around the counter.

Please refer to Figure 1. The capture module 2 is electrically connected to the camera device 1 . The capturing module 2 can perform color space conversion, color filtering and edge detection according to a block to obtain a face image. For details, please refer to FIG. 3 together. The system administrator can select a Region of Interest ROI from the monitoring screen in advance as the aforementioned block, thereby saving the image capturing cost of the capturing module 2 for images other than the ROI of the region of interest. In practice, the ROI of the region of interest in the screen can be preset according to the perspective of the counter service staff, as shown in Figure 3.

After setting the block according to the region of interest ROI, the capture module 2 can convert the RGB color space of the block into one of HSV color space, CIE 1931 color space, YIQ color space and YCbCr color space, thereby reducing light The impact on subsequent face detection. After the color space conversion is completed, the capture module 2 separates the skin color from the background color. In detail, the distribution of skin tones in the transformed color space can be defined by a specified set of equations. In addition, since the color of the background objects in the business hall of the financial institution is fixed, the background color can be defined by another set of specified equations. The capture module 2 uses, for example, Otsu Thresholding or Balanced Histogram Thresholding (BHT) to find out all the pixels in the block that meet the skin color conditions. And filter out the pixels with background color to speed up the overall processing speed.

In one embodiment, for the binarized image, the capturing module 2 may employ a morphological dilation operator and an erosion operator to eliminate noise outside the face area. Specifically, the capture module 2 may perform a Closing operation of dilation and then erosion, and then perform an Open operation of erosion and then dilation, so as to highlight the contours around the facial features in the image.

For the image block after noise removal, the capturing module 2 selects the face image in the block according to the geometric model of the face. The geometric model can be defined according to the ratio of the height to the width of a general face, for example: 0.79≤face height/face width≤2.95.

In another embodiment, the capture module 2 uses Canny filter, Sobel filter or Prewitt operator, for example, to perform edge detection of the face image, and further output a person face image.

In yet another embodiment, the capturing module 2 further divides the block image based on the YUV color coding method into a plurality of overlapping small areas, and calculates each small area for the brightness value (ie the Y element) in these areas. The edge intensity values of the area in the horizontal direction, the vertical direction and the two diagonal directions are compared with each other according to the calculated intensity values and a threshold value to obtain the edge of the face, and further output the face image.

In yet another embodiment, the capturing module 2 compares the image samples of eigenfaces trained in advance, and then outputs a face image. In practice, financial institutions can perform principal component analysis (Principal Component Analysis, PCA) on image files with different faces retained by many customers when opening accounts to obtain a set of eigenface vectors as the aforementioned image samples.

In practice, the capture module 2 is, for example, a digital signal processor (digital signal processor), an application specific integrated circuit (ASIC), a digital logic circuit, and a field programmable gate array (field programmable gate array, FPGA) or other hardware components that can perform the above-mentioned face image capture function.

The image processing module 3 is electrically connected to the capturing module 2 . The image processing module 3 obtains a plurality of feature points according to the face image, captures sub-images according to the distribution of these feature points, and then calculates measurable information according to the sub-images.

In one embodiment, the image processing module 3 uses a connected-component labeling algorithm to find connected pixels in the image, so as to mark the coordinates of the facial features in the face image as feature points.

In another embodiment, the image processing module 3 uses an Active Shape Model (Active Shape Model) or an Active Appearance Model (Active Appearance Models, AAM) to calculate the coordinates of the facial features in the face image at one time.

The sub-image captured by the image processing module 3 according to the feature points includes one of the user's facial features and surrounding skin. In one embodiment, the sub-image is a first sub-image, and the first sub-image includes the user's eye and surrounding skin. In another embodiment, the image processing module 3 further captures the second sub-image, the third sub-image, the fourth sub-image and the fifth sub-image respectively according to the face image. The second sub-image includes the user's eyebrows and its surrounding skin, the third sub-image includes the user's nose and its surrounding skin, the fourth sub-image includes the user's mouth and its surrounding skin, and the fifth sub-image includes the user's skin. Ear.

After obtaining the sub-image, the image processing module 3 calculates measurable information according to the sub-image. The measurable information includes a plurality of feature points to form a straight line or a region or the area of the region. Measurable information includes eye shape, eye distance, eyebrow length, eyebrow distance, nose width, nose bridge length, upper lip thickness, lower lip thickness, ear shape, ear color, and the length of wrinkles, depth of wrinkles and skin around the facial features Wait for class.

The face database 4 includes a plurality of first table spaces, each of which corresponds to one of the facial features and includes a plurality of first fields. Specifically, each tablespace corresponds to one of the above-mentioned types of measurable information. For example, tablespace A is associated with eye shape and tablespace B is associated with binocular distance. Each first field stores the attribute, evaluation value and a weight value of the measurable information. Attributes include the length of the line, the shape and color of the area. Following the previous example, the attributes stored in each field of table space A correspond to different eye shapes, such as: almond eye, red phoenix eye, slanted eye, slender eye, round eye, droopy eye, triangular eye, etc. Each eye shape corresponds to to an estimated value. The attributes stored in each field of table space B correspond to different distances between eyes, such as 1.1~1.3 cm, 1.3~1.5 cm, 1.5~1.7 cm, etc.

The respective evaluation values and weight values of the plurality of first fields in the face database 4 are obtained by running a deep learning algorithm through another computing module in advance. In detail, financial institutions can use the original customer image database, investor risk attribute analysis questionnaires filled in by customers, and financial product projects invested by customers as the original training set. Network, RNN) or Long Short Term Memory (Long Short Term Memory, LSTM) to obtain the specific facial features corresponding to various financial product types, and then calculate the evaluation value and weight value stored in each field. The evaluation value is related to the risk proportion that investors are willing to bear, and the weight value represents the proportion of each type when considering various measurable information of facial features.

The financial product database 5 includes a plurality of second table spaces. The second table spaces respectively correspond to a plurality of risk ratio ranges and include a plurality of second fields, each of which stores a financial product and an age range. For example, the corresponding risk ratio range of table space C is 0~6, and its risk attribute is typically classified as "conservative", which means that investors are basically unwilling to take any investment risks, and suitable financial products tend to be rewarded from interest income. Product; the corresponding risk ratio range of table space D is 7~13, and its risk attribute is typically classified as "very cautious", which means that investors can basically accept slight losses in exchange for slight potential investment returns. Suitable financial products Projects rated as low risk for financial institutions. It should be noted that the boundary value of the above risk ratio interval is obtained by running a deep learning algorithm through another computing module in advance.

The computing module 6 is communicatively connected to the image processing module 3 , the face database 4 and the financial product database 5 . The computing module 6 estimates an age of the user according to the measurable information. To be more specific, the computing module 6 mainly estimates the age based on measurable information such as the wrinkle length, wrinkle depth, and skin point of the skin around the facial features, especially the forehead lines above the eyebrows, the nasolabial lines on both sides of the nose, and Crow's feet around the eyes. The image processing module 3 uses Canny edge detector technology to detect the texture of wrinkles, and sets appropriate thresholds for age judgment. On the other hand, since the skin of the human face presents different features with aging, the skin color of the skin area of the human face image can be segmented as an auxiliary method for identifying the age. Specifically, by properly selecting the parameter thresholds in the Y, C _b and C _r color spaces, and detecting the face image output by the capture module 2 , age spots can be detected. For example, if a specific block of the face image conforms to the following formula: (24≤Y≤124) and (113≤C _b ≤140) and (115≤C _r ≤143), it can be determined that the specified block has Age spots. The computing module 6 can further estimate the age of the user according to the color and number of age spots and the number of forehead lines, nasolabial lines and crow's feet.

The operation module 6 obtains one or more evaluation values from the face database 4 according to the attributes corresponding to the measurable information of the five senses, calculates the risk value according to the evaluation value, and obtains the risk value according to one of the risk ratio intervals corresponding to the risk value. A second table space is selected in the product database 5, and a second field is selected and output from the selected second table space according to the age range corresponding to the age. In practice, the computing module 6 obtains five evaluations from the face database 4 according to the attributes corresponding to the measurable information of the first sub-image, the second sub-image, the third sub-image, the fourth sub-image and the fifth sub-image. value and the five-weight value, and the risk value is calculated according to the five-assessment value and the five-weight value. In detail, the computing module 6 first obtains the risk assessment values according to the features of the user's facial features (for example, the corresponding risk assessment values for the eyes, eyebrows, nose, and ears are A ₁ , A ₂ , A ₃ , A ₄ and A ₅ ), and then according to the corresponding weight values of the five sense attributes (for example, A ₁ corresponds to W ₁ , A ₂ corresponds to W ₂ ..., A ₅ corresponds to W ₅ ), and then these risk assessment values are summed up according to their corresponding weight values Obtain a comprehensive risk assessment value for the user (eg A ₁ *W ₁ +A ₂ *W ₂ +A ₃ *W ₃ +A ₄ *W ₄ +A ₅ *W ₅ ). The computing module 6 obtains a corresponding list of financial products from the financial product database 5 according to the comprehensive risk assessment value, and then obtains a recommended financial product according to the user age estimated by the computing module.

Based on the above, the image recognition system disclosed in the present invention can quickly lock a face image in a specific area from a complex image captured by a surveillance camera, and analyze the age and facial features of the face image through face recognition. The measurable information of , and then convert the multiple measurable information into an evaluation score. Therefore, when a customer visits a specific counter in a business hall of a financial institution, the image recognition system disclosed in this case can further recommend suitable financial products based on the range of evaluation scores.

Although the present invention is disclosed in the foregoing embodiments, it is not intended to limit the present invention. Changes and modifications made without departing from the spirit and scope of the present invention belong to the scope of patent protection of the present invention. For the protection scope defined by the present invention, please refer to the attached patent application scope.

10: Image recognition system 1: Camera device 2: Capture module 3: Image processing module 4: Face database 5: Financial product database 6: Operation module ROI: Region of Interest

FIG. 1 is a structural diagram of an image recognition system according to an embodiment of the present invention. 2 is a schematic top view of a camera device installed in a business hall of a financial institution according to an embodiment of the present invention. FIG. 3 is a schematic diagram of an image captured by a camera device according to an embodiment of the present invention.

10: Image recognition system

1: Camera device

2: Capture module

3: Image processing module

4: Face database

5: Financial product database

6: Operation module

Claims (2)

An image recognition system, comprising: a camera device for acquiring a single image; a capture module for communicating with the camera device, the capture device for selecting a block from the image and obtaining a use in the block A face image of the above, wherein the capture module is further used to perform color space conversion, color filtering and edge detection according to the block to obtain the face image, and the capture module is further used to convert the block A color space is one of HSV color space, CIE 1931 color space, YIQ color space and YCbCr color space; an image processing module, communicatively connected to the capture module, and the image processing module captures the face image according to the face image A sub-image calculates a measurable information according to the individual sub-image, wherein the individual sub-image includes one of the facial features of the user and the surrounding skin, and the measurable information includes a straight line formed by a plurality of feature points and an area; a photo database, including a plurality of first table spaces, each of the first table spaces corresponds to one of the facial features and includes a plurality of first fields, each of the first fields stores An attribute and an evaluation value of the measurable information, wherein the attribute includes a length of the straight line, a shape and a color of the area; a financial product database includes a plurality of second table spaces, the second The table space corresponds to a plurality of risk ratio intervals and includes a plurality of second fields, each of the second fields stores a financial product and an age interval; In the face database and the financial product database, the computing module estimates an age of the user according to the measurable information, and obtains the evaluation value from the face database according to the attribute corresponding to the measurable information, Calculate a risk value according to the assessment value, select one of the second table spaces from the financial product database according to one of the risk ratio intervals corresponding to the risk value, and select the age corresponding to the age The interval selects a second field from the selected second table space and outputs, wherein the sub-image is a first sub-image, and the first sub-image includes the user's eye and surrounding skin; the image processing The module further captures a second sub-image, a third sub-image, a fourth sub-image and a fifth sub-image according to the face image, and the second sub-image includes the user's eyebrow and surrounding skin , the third sub-image includes the user's nose and its surrounding skin, the fourth sub-image includes the user's mouth and its surrounding skin, the fifth sub-image includes the user's ear; the measurable information It also includes eye shape, eye distance, eyebrow length, eyebrow distance, nose width, nose bridge length, upper lip thickness, lower lip thickness, ear shape and ear color; each of the first fields of the face database further stores the A weight value of the measurable information; the operation module is based on the measurable information of the first sub-image, the second sub-image, the third sub-image, the fourth sub-image and the fifth sub-image respectively The corresponding attribute obtains five evaluation values and five weight values from the face database, and calculates according to the five evaluation values and the five weight values. Calculate the risk value; and the evaluation values and the weight values of the first fields of the face database, and the boundary values of the risk ratio intervals of the financial product database, are pre-processed by another calculation The module is obtained by running a deep learning algorithm; wherein the capturing module is further used to perform edge detection according to the Canny operator, the Sauber operator or the Prewitt operator to output the face image; wherein the deep learning algorithm The law is a long short-term memory neural network or a recurrent neural network, and the input layer of the deep learning algorithm includes: customer image database, investor risk attribute analysis questionnaire and investment financial product projects, and the deep learning algorithm The output layer includes the evaluation value and the weight value stored in each of the first fields. The image recognition system according to claim 1, wherein the capturing module compares an eigenface sample to output the human face image.

Images