TWM593011U - Image recognition system - Google Patents

Abstract

An image recognition system comprises two camera devices, an image processing device, a posture database, a financial product database, and a computing device. Said two camera devices respectively obtain an image of a front view and an image of a side view. The image processing device selects a plurality of positioning points from the image of the front view and the side view and calculates a length, a slope, and a connection type of two of these positioning points. Each tablespace of the posture database corresponds to a posture type and comprises a plurality of fields. Each field stores a posture value and a risk evaluation value. Each tablespace of the financial product database corresponds to an interval of a risk ratio and stores the information of a financial product. The computing device calculates an integrated risk evaluation value and outputs the information of the financial product.

Description

Image recognition system

This creation is about an image recognition system, especially an image recognition system that obtains human posture information from images.

For example, financial institutions such as banks and post offices set up multiple surveillance cameras in their business halls to monitor people traveling in and out of the business halls.

However, the surveillance images taken by these surveillance cameras are usually used to review afterwards special events that occurred at a specific point in time (such as masked robbery, driver theft, etc.). In other words, the monitoring system set up by financial institutions lacks a mechanism for instant processing and reporting based on current images. Although the monitoring system has been running for a long time, it is only used as a safety device to deter people who are interested. In addition, the overhead surveillance screen from a height must include all kinds of people and objects, such as waiting people, counter people, counter clerks, lobby floors, waiting seats, ATMs, folding machines, etc.; even if Arranging a dedicated person to watch the surveillance image and quickly lock the posture of a specific person from a complex picture is not easy, let alone that humans are prone to miss important information in the picture due to fatigue or distraction.

In addition, for high-net-worth customers who frequently come and go, financial institutions provide exclusive financial services to customers at this customer level. Through customized commodity investment portfolios, high-net-worth customers can increase their wealth. On the other hand, financial institutions usually assist clients who are generally willing to carry out wealth management through mid-term and long-term asset planning through medium- and long-term asset planning. By understanding the client’s current financial flow and future capital use planning, they conduct risk assessments to propose a complete financial plan. Product recommendation plan.

However, for customers who simply come to the business hall to deal with general financial business, the clerks or financial specialists of the financial institution cannot grasp whether the customer is a potential customer with financial needs based on the posture of the customer. Therefore, when the clerks sell financial products It is often formalized, unable to provide financial products that may meet the needs of this type of customer, and can only blindly promote projects that the customer is not interested in, wasting both parties' time and energy in vain.

In view of this, this author proposes an image recognition system that can obtain the user's full-body image from images captured by multiple surveillance cameras with different shooting angles, and further calculate the measurable information of the body posture. Using these posture values to analyze in a pre-trained database can increase the sales success rate of financial products and reduce the chances of customers being emotionally unsatisfactory because they are recommended by unsuitable products.

An image recognition system according to an embodiment of the present invention includes a first camera device, a second camera device, an image processing device, a posture database, a financial product database, and a computing device. The first camera device obtains a frontal image of the user while standing. The second camera device obtains a side image of the user while standing. The image processing device is electrically connected to the first imaging device and the second imaging device. The image processing device selects a plurality of positioning points related to the user from the front image, and calculates the length of two of the positioning points. The image processing device selects a plurality of positioning points related to the user from the side image, and calculates the slope and connection type of two of the positioning points. The posture database includes a plurality of first table spaces. Each first table space corresponds to the integrated state type. Each first table space includes a plurality of first fields. Each first field stores the posture value and risk assessment value of the posture type. The financial product database includes a plurality of second table spaces. Each second table space corresponds to a risk ratio interval and stores a financial product information. The computing device communicates with the image processing device, posture database and financial product database. The computing device obtains a plurality of risk assessment values from the posture database according to the length, slope and connection type. The computing device calculates a comprehensive risk assessment value based on the obtained risk assessment values. The computing device selects a second table space from the financial product database to output the corresponding financial product information, wherein the comprehensive risk assessment value belongs to the risk ratio interval corresponding to the selected second table.

With the above structure, the image recognition system disclosed in this case can obtain the front and side images of the user from multiple camera devices installed in the business hall of the financial institution, and obtain the posture information of the human body in the image through image processing technology, and will Multiple posture information is converted into a risk assessment score. Therefore, when a customer visits a specific counter in the business hall of a financial institution, the image recognition system disclosed in this case can further recommend financial products suitable for the customer based on the range of the comprehensive risk assessment value, thereby giving the customer a good financial service experience . In addition, the posture information measured by the image recognition system created by the author can also be aggregated for customers' reference.

The above description of the content of the disclosure and the following description of the embodiments are used to demonstrate and explain the spirit and principle of the creation, and provide a further explanation of the scope of the patent application of the creation.

The following describes in detail the detailed features and advantages of the creation in the embodiments. The content is sufficient for any person skilled in the relevant art to understand and implement the technical content of the creation, and according to the content disclosed in this specification, the scope of patent application and the drawings Anyone who is familiar with related skills can easily understand the purpose and advantages of this creation. The following examples further illustrate the views of this creation in detail, but do not limit the scope of this creation with any views.

Please refer to FIG. 1, which is a block diagram of an image recognition system 10 according to an embodiment of the present invention. The image recognition system 10 can be installed in the business hall of a financial institution, and is suitable for users (customers) standing at designated counters who are expected to handle financial-related businesses. The image recognition system 10 includes: a first camera device 1, a second camera device 2, an image processing device 3, a posture database 4, a financial product database 5 and an arithmetic device 6.

Please refer to FIG. 2, which is a schematic top view of the first camera device 1 and the second camera device 2 installed in the business hall of a financial institution. As shown in FIG. 2, the installation position of the first camera 1 can capture a frontal image of a user standing in front of the designated counter in the business hall of the financial institution, and the installation position of the second camera 2 can be photographed in the business hall of the financial institution A silhouette of a user standing in front of a designated counter. In practice, two second camera devices 2 may be provided to capture the left and right images of the user, respectively, and the number of second camera devices 2 is not limited in this creation.

Please refer to Figure 1. The image processing device 3 is electrically connected to the first imaging device 1 and the second imaging device 2. The image processing device 3 can search for the presence of the user in the environment through the image information such as the front image and the side image captured by the first camera device 1 and the second camera device 2, and complete the recognition of the user's posture. In detail, the image processing device 3 first performs color space conversion, color filtering, and edge detection for the front image and the side image, respectively.

The image processing device 3 can convert the RGB color space of the front image and the side image into one of the HSV color space, the CIE 1931 color space, the YIQ color space, and the YCbCr color space, thereby reducing the light for subsequent human body contour detection influences.

After the color space conversion is completed, the image processing device 3 extracts skin color image blocks from the original front image and side image, and simultaneously filters the background color blocks. Specifically, the image processing device 3 defines a specified equation group to represent the distribution of skin color after converting the color space, and defines another specified equation system to represent the color distribution of the furnishings in the business hall of the financial institution. In practice, the image processing device 3 uses, for example, a Gaussian Mixture Model (GMM) to create a description equation of the color distribution of the background object, and then uses background subtraction or adaptive background subtraction Obtain foreground information, and then perform vertical pixel projection on this foreground information. If the peak value of the histogram is within a reasonable height range of the human body, it indicates that there may be someone there. The image processing device 3 uses, for example, Otsu Thresholding or Balanced Histogram Thresholding (BHT) to find all pixels that match the skin color distribution from the image block of a possible person.

In an embodiment, for the binarized image, the image processing device 3 may use a dilation operator and an erosion operator on the morphology to eliminate noise outside the skin color region. Specifically, the image processing device 2 may perform a closing operation of expanding first and then eroding (Closing operation) and then an opening operation of first eroding and then expanding (Operation operation), thereby highlighting the contour around the skin color block in the image.

After acquiring the images of all skin-color blocks, the image processing device 3 uses a connected-component labeling algorithm to find the pixels connected in these skin-color block images, and obtain the length and width of each block and coordinate. The image processing device 3 further performs elliptical comparison for each block, thereby obtaining the skin color blocks of the human head and the limbs not covered by the underwear. The image processing device 3 uses, for example, one of the Canny filter, Sobel filter and Prewitt operator to perform edge detection on the skin color block to obtain the positioning points of the human head and limbs. After obtaining the above-mentioned positioning points, the image processing device 3 can calculate a plurality of positioning points associated with the user in the front image and associated with the user in the side image based on the foreground image previously obtained by using the background subtraction method and the human skeleton database Multiple anchor points.

The image processing device 3 obtains at least two positioning points from the plurality of positioning points of the front image and calculates its length. For example, the height is calculated according to the positioning point of the human head and the positioning point of the human foot, and the slope of each straight line is calculated according to the polyline of multiple positioning points of the human shoulder and back and the overall connection type is determined.

In practice, the image processing device 3 such as a coefficient signal processor (digital signal processor), an application specific integrated circuit (ASIC), a digital logic circuit, and a field programmable gate array (FPGA) ) Or other hardware components that can perform the above image processing functions, this author does not limit this.

In another embodiment of the image recognition system 10 of the present invention, in addition to the first camera device 1 and the second camera device 2, a third camera device (not shown) is further included. The third imaging device is installed, for example, on the ceiling of the business hall of the financial institution, and is electrically connected to the image processing device 3. The third camera device is used to obtain an overhead image of the user. The image processing device 3 captures the skin color and black color blocks from the overhead image and calculates the ratio of the two, and at the same time determines which type of male baldness (Norwood’s Classification) the shape of the black block belongs to.

In yet another embodiment of the image recognition system 10 of the present invention, in addition to the first camera device 1 and the second camera device 2, a fourth camera device (not shown) is further included. The fourth imaging device is electrically connected to the video processing device 3. The installation location of the fourth camera 1 can capture the back image of the user standing in front of the designated counter in the business hall of the financial institution. The first camera 1 and the fourth camera are depth cameras, so the front image captured by the first camera 1 further includes first depth information, and the back image captured by the fourth camera further includes second depth information. The installation positions of the first camera 1 and the fourth camera form a first connection. The standing position of the user and the installation position of the first camera device form a second connection. The standing position of the user and the installation position of the fourth camera form a third connection. The first connection and the second connection form a first angle, and the first connection and the third connection form a second angle. In this embodiment, the image processing device 3 is further used to calculate another posture value of another posture type based on the first depth information, the second depth information, the length of the first connection, the first included angle, and the second included angle . Specifically, since the length of the first connection is a fixed value, the image processing device 3 can calculate the number of the first camera device 1 to the front of the user's body based on the first included angle, the first depth information, and basic trigonometric function calculations A distance, and the second distance from the fourth camera to the back of the user’s body can be calculated based on the second included angle, the second depth information, and basic trigonometric calculations, and then the length of the first connection is deducted from the first distance and the first Two distances can obtain the thickness information of the user's body under the designated positioning point.

The posture database 4 includes a plurality of first table spaces, and each first table space corresponds to a posture type and includes a plurality of first fields. The posture types include height, shoulder width, arm length, leg length, hair volume, hair rotation, male baldness stage, hunchback degree, side standing position type, body thickness, etc. This creation does not limit the types and number of posture types .

Each first field of each first table space stores an integrated evaluation value, a weight value, and a risk evaluation value. The risk assessment value is related to the proportion of risk that investors are willing to bear. The weight value is used to determine the proportion of each posture type when evaluating multiple posture types. For example, the table space corresponding to height is shown in the following table:

The table space corresponding to the side stance is shown in the following table:

In an embodiment of the present creation, the posture evaluation values (or posture evaluation types), weight values, and risk evaluation values listed in the above table are obtained by training in advance through a computing device based on a historical database. The historical database includes financial institutions' positive images, side images, and investor risk attribute analysis questionnaires filled out by customers based on the accounts opened by customers, and financial product investment records accumulated by customers during the course of conducting business with financial institutions, etc. Big Data. The aforementioned training, specifically, the computing device executes a deep learning algorithm and a classifier algorithm. The deep learning algorithm is, for example, one of a Recursice Neural Network (RNN) and a Long Short Term Memory (LSTM). The aforementioned classifier algorithm is, for example, one of a support vector machine (SVM) and a multi-layer perceptron (MLP). Through the pre-training of the computing device, you can get the posture types corresponding to the normality of various financial products, and then calculate the posture evaluation value, weight value and risk evaluation value stored in each field of each table space.

Please refer to Figure 1. The financial product database 5 includes a plurality of second table spaces. Each second table space corresponds to a risk ratio interval and stores a financial product information. For example, the "conservative" financial product corresponds to a risk ratio range of 0~6, which represents that investors are basically reluctant to bear any investment risk, so the financial product information recorded in this table space is biased in compensation from interest income The product. The "prudential" financial product corresponds to a risk ratio range of 7 to 13, which represents that investors can basically accept slight losses in exchange for slight potential investment returns. Therefore, the financial product information recorded in this table space is financial The agency rated it as a low-risk project. It should be noted that the boundary value of the above risk ratio interval can also be obtained by running the deep learning algorithm and classifier algorithm through the aforementioned computing device in advance, or defined by the relevant person in charge of the financial institution. To be restricted.

Please refer to Figure 1. The computing device 6 communicates with the image processing device 3, the posture database 4 and the financial product database 5.

The computing device 6 obtains one or several risk assessment values from the posture database 4 according to the posture information such as the length, slope and connection type measured by the image processing device 3, and calculates a comprehensive risk assessment value based on these risk assessment values Then, select a second table space in the financial product database 5 corresponding to the risk ratio interval corresponding to the comprehensive risk assessment value and output the corresponding financial product information. For example, the computing device 6 may search the posture database 4 for the second risk evaluation value and the second weight value according to the posture information such as height and side standing posture, and calculate the comprehensive risk evaluation value based on the second risk evaluation value and the second weight value. Following the example in the previous table, the risk assessment value corresponding to height 165 is 0.37 and the weight value is 1. The risk assessment value corresponding to the user with a straight back profile is 0.53 and the weight value is 1.23. The computing device 6 adds up these risk assessment values according to their corresponding weight values to obtain a comprehensive risk assessment value of the user (for example, 0.37*1+0.53*1.23=1.0219). The computing device 6 obtains and outputs corresponding financial product information in the financial product database 5 based on the comprehensive risk assessment value.

Based on the above, the image recognition system disclosed in this creation can obtain the front and side images of the user from multiple camera devices installed in the business hall of the financial institution, and obtain the posture information of the human body in the image through image processing technology, and Convert multiple posture information into risk assessment scores. Therefore, when a customer visits a specific counter in the business hall of a financial institution, the image recognition system disclosed in this case can further recommend financial products suitable for the customer based on the range of the comprehensive risk assessment value, thereby giving the customer a good financial service experience . In addition, the posture information measured by the image recognition system created by the author can also be aggregated for customer reference.

Although this creation is disclosed above with the foregoing embodiments, it is not intended to limit this creation. Without departing from the spirit and scope of this creation, all changes and retouching are within the scope of patent protection of this creation. For the protection scope defined by this creation, please refer to the attached patent application scope.

10: Image recognition system 1: The first camera 2: Second camera 3: Image processing device 4: Posture database 5: Financial product database 6: computing device

FIG. 1 is a block diagram of an image recognition system according to an embodiment of the present invention. FIG. 2 is a configuration diagram of a camera device shown in an image recognition system according to an embodiment of the present invention.

10: Image recognition system

1: The first camera

2: Second camera

3: Image processing device

4: Posture database

5: Financial product database

6: computing device

Claims (8)

An image recognition system includes: a first camera device for obtaining a front image of a user while standing; a second camera device for obtaining a side image of the user while standing; an image processing device, The first camera device and the second camera device are electrically connected, and the image processing device is used to select a plurality of positioning points associated with the user from the front image and calculate a length of two of the positioning points, And select a plurality of anchor points associated with the user from the profile and calculate a slope and a connection type of two anchor points of the anchor points; the integrated database includes a plurality of first table spaces, each The first table spaces correspond to the integrated state type and include a plurality of first fields, and each of the first fields stores an evaluation value, a risk value, and a weight value of the body type; a financial product database , Including a plurality of second table spaces, each of which corresponds to a risk ratio interval and stores financial product information; and a computing device that communicates with the image processing device, the posture database, and the financial product Database, the computing device obtains a plurality of the risk assessment values from the posture database according to the length, the slope and the connection type, and calculates a comprehensive risk assessment value based on the acquired risk assessment values, And select one of the second table spaces from the financial product database to output the corresponding financial product information, wherein the comprehensive risk assessment value belongs to the selected risk ratio interval corresponding to the selected second table. The image recognition system according to claim 1, further comprising a third camera device electrically connected to the image processing device, the third camera device is used to obtain an overhead image of the user The image processing device is further used to select a plurality of positioning points from the overhead image and calculate a block formed by the positioning points and a color distribution of the block. The image recognition system according to claim 1, further comprising: a fourth camera device electrically connected to the image processing device, the fourth camera device used to obtain a back image of the user while standing; wherein, the first A camera device and the fourth camera device are depth cameras. The first camera device and the fourth camera device form a first connection. The front image further includes a first depth information, and the back image further includes a first depth information. A second depth information; the image processing device is further used to calculate another body value of another body type according to the first depth information, the second depth information, and the length of the first connection. The image recognition system according to claim 3, wherein the standing position of the user and the installation position of the first camera form a second connection; the standing position of the user and the installation position of the fourth camera form A third connection; the first connection and the second connection form a first angle; the first connection and the third connection form a second angle; and the image processing device further includes An angle and the second angle calculate the other body value. The image recognition system according to claim 1, wherein the computing device is a first computing device, and further includes a second computing device to communicate with the posture database and the financial product database, and the second computing device is used to Run a deep learning algorithm and a classifier algorithm based on a historical database to update the posture value and the risk assessment value to the posture database and update The risk ratio ranges to the financial product database, wherein the historical database is used to store the front image, the side image and the financial product investment records of each of the multiple customers. The image recognition system according to claim 5, wherein the deep learning algorithm is one of a long-short-term memory neural network or a recursive neural network, and the classifier algorithm is one of a multi-layer perceptron or a support vector machine. The image recognition system according to claim 1, wherein the image processing module is further used for color space conversion, color filtering and edge detection according to the front image and the side image. The image recognition system according to claim 1, wherein the image processing device is further used to convert one color space of the front image and the side image into the HSV color space, the CIE 1931 color space, the YIQ color space, and the YCbCr color space One of them.

Images