TWI758904B - Image recognition system having multiple cameras - Google Patents

Abstract

An image recognition system comprises two camera devices, an image processing device, a posture database, a financial product database, and a computing device. Said two camera devices respectively obtain an image of a front view and an image of a side view. The image processing device selects a plurality of positioning points from the image of the front view and the side view and calculates a length, a slope, and a connection type of two of these positioning points. Each tablespace of the posture database corresponds to a posture type and comprises a plurality of fields. Each field stores a posture value and a risk evaluation value. Each tablespace of the financial product database corresponds to an interval of a risk ratio and stores the information of a financial product. The computing device calculates an integrated risk evaluation value and outputs the information of the financial product.

Description

Image recognition system with multiple cameras

The present invention relates to an image recognition system, in particular to an image recognition system for obtaining human body posture information from images.

For example, banks, post offices and other financial institutions have set up multiple surveillance cameras in their business halls to monitor the people moving in and out of the business halls.

However, the surveillance images captured by these surveillance cameras are often used to review special events (such as masked robberies, driver theft, etc.) that occurred at a specific point in time after the event. That is to say, the surveillance system set up by financial institutions lacks a mechanism for real-time processing and reporting based on current images. Although the surveillance system runs for a long time, it only serves as a safety device to deter people with intent. Furthermore, the overhead surveillance footage from a height must include all kinds of people and objects, such as waiting people, people at the counter, counter clerks, hall floors, waiting seats, ATMs, refilling machines, etc.; It is not easy to arrange a special person to watch the surveillance video and quickly locate the posture of a specific person from a complex picture, not to mention that human beings are prone to miss important information in the picture due to fatigue or distraction.

In addition, for frequent high-net-worth clients, financial institutions provide exclusive financial services for clients of this segment, and allow high-net-worth clients to increase their wealth through customized commodity investment portfolios. On the other hand, financial institutions usually assist clients who are willing to carry out wealth management to make medium and long-term asset planning through their extensive marketing bases, and conduct risk assessment by understanding the current cash flow of clients and their future capital utilization plans, so as to propose a complete financial plan. Product Recommendation Program.

However, for customers who simply come to the business hall to handle general financial business, the staff or wealth management specialists of financial institutions cannot know whether the customer is a potential customer with wealth management needs based on the customer's posture. Therefore, when the staff sells financial products It is often only formalized, unable to provide financial products that may meet the needs of this type of customers, and can only blindly sell projects that customers are not necessarily interested in, wasting time and energy of both parties.

In view of this, the present invention proposes an image recognition system with multiple cameras, which can obtain a full-body image of a user from images captured by a plurality of surveillance cameras with different shooting angles, and further calculate measurable information of body posture . Using these posture values to analyze the pre-trained database can improve the sales success rate of financial products and reduce the probability of customers being in bad mood due to recommended products that are not suitable.

An image recognition system with multiple camera devices according to an embodiment of the present invention includes a first camera device, a second camera device, an image processing device, a posture database, a financial product database, and a computing device. The first camera device obtains a frontal image of the user standing. The second camera device obtains a side image of the user standing. The image processing device is electrically connected to the first camera device and the second camera device. The image processing device selects a plurality of positioning points associated with the user from the frontal image, and calculates the length of two positioning points among the positioning points. The image processing device selects a plurality of positioning points associated with the user from the silhouette image, and calculates the slope and the connection type of two positioning points among the positioning points. The posture database includes a plurality of first table spaces. Each first tablespace corresponds to a state type. Each first tablespace includes a plurality of first fields. Each first field stores the posture assessment value and the risk assessment value of the posture type. The financial product database includes a plurality of second table spaces. Each second table space corresponds to a risk ratio interval and stores information of a financial product. The computing device is communicatively connected to the image processing device, the posture database and the financial product database. The computing device obtains a plurality of risk assessment values from the posture database according to the length, slope and connection type. The computing device calculates a comprehensive risk assessment value according to the obtained risk assessment values. The computing device selects a second table space from the financial product database to output corresponding financial product information, wherein the comprehensive risk assessment value belongs to the risk ratio range corresponding to the selected second table.

With the above structure, the image recognition system disclosed in this case can obtain the front and side images of the user from a plurality of camera devices installed in the business hall of the financial institution, and obtain the body posture information of the human body in the image through image processing technology, and Multiple body information is converted into risk assessment scores. Therefore, when a customer visits a specific counter in the business hall of a financial institution, the image recognition system disclosed in this case can further recommend financial products suitable for the customer according to the range of the comprehensive risk assessment value, thereby allowing the customer to have a good financial service experience . In addition, the body posture information measured by the image recognition system of the present invention can also be collected for reference by customers.

The above description of the present disclosure and the following description of the embodiments are used to demonstrate and explain the spirit and principle of the present invention, and provide further explanation of the scope of the patent application of the present invention.

The detailed features and advantages of the present invention are described in detail below in the embodiments, and the content is sufficient to enable any person skilled in the relevant art to understand the technical content of the present invention and implement it accordingly, and according to the content disclosed in this specification, the scope of the patent application and the drawings , any person skilled in the related art can easily understand the related objects and advantages of the present invention. The following examples further illustrate the viewpoints of the present invention in detail, but do not limit the scope of the present invention in any viewpoint.

Please refer to FIG. 1 , which is a block diagram of an image recognition system 10 according to an embodiment of the present invention. The image recognition system 10 can be installed in the business hall of a financial institution, and is suitable for users (customers) who are expected to handle financial-related business while standing at a designated counter. The image recognition system 10 includes: a first camera device 1 , a second camera device 2 , an image processing device 3 , a posture database 4 , a financial product database 5 and a computing device 6 .

Please refer to FIG. 2 , which is a schematic top view of the first camera device 1 and the second camera device 2 installed in the business hall of a financial institution. As shown in FIG. 2 , the installation position of the first camera device 1 can capture the frontal image of the user standing in front of the designated counter in the business hall of the financial institution, and the installation position of the second camera device 2 can take pictures in the business hall of the financial institution Silhouette of a user standing in front of a designated counter. In practice, two second camera devices 2 can be set to capture the left image and the right image of the user respectively, and the present invention does not limit the number of the second camera devices 2 .

Please refer to Figure 1. The image processing device 3 is electrically connected to the first camera device 1 and the second camera device 2 . The image processing device 3 can search for the existence of the user in the environment and complete the identification of the user's posture by using the image information such as the front image and the side image captured by the first camera device 1 and the second camera device 2 . Specifically, the image processing device 3 firstly performs color space conversion, color filtering and edge detection for the front image and the side image respectively.

The image processing device 3 can convert the RGB color space of the front image and the side image into one of the HSV color space, the CIE 1931 color space, the YIQ color space, and the YCbCr color space, thereby reducing the impact of light on subsequent human contour detection. Influence.

After the color space conversion is completed, the image processing device 3 extracts the image blocks of skin color from the original frontal image and the side image, and simultaneously filters the blocks of background color. Specifically, the image processing device 3 defines a specified equation set to represent the skin color distribution after the color space conversion, and defines another specified equation set to represent the color distribution of the furnishings in the business hall of the financial institution. In practice, the image processing device 3 uses, for example, a Gaussian Mixture Model (GMM) to establish a description equation for the color distribution of background objects, and then uses background subtraction or adaptive background subtraction. Obtain foreground information, and then perform pixel vertical projection on this foreground information. If the peak value of the histogram is within the range of reasonable body height, it means that there may be someone there. The image processing device 3 uses, for example, the Otsu Thresholding method or the Balanced Histogram Thresholding (BHT) method to find out all the pixels matching the skin color distribution from the image blocks that may be human.

In one embodiment, for the binarized image, the image processing device 3 may employ a morphological dilation operator and an erosion operator to eliminate noise outside the skin color area. Specifically, the image processing device 2 may perform a Closing operation of dilation followed by erosion, and then perform an Open operation of erosion followed by dilation, so as to highlight the contours around the skin color area in the image.

After acquiring the images of all the skin color blocks, the image processing device 3 uses a connected-component labeling algorithm to find out the connected pixels in these skin color block images, and obtains the length and width of each block and the coordinate. The image processing device 3 further performs ellipse comparison for each block, thereby obtaining the skin color blocks of the human head and the limbs not covered by clothes. The image processing device 3 uses, for example, one of the Canny filter, the Sobel filter and the Prewitt operator to perform edge detection on the skin color block to obtain the positioning points of the human head and limbs. After obtaining the above positioning points, the image processing device 3 can calculate a plurality of positioning points related to the user in the frontal image and a plurality of positioning points related to the user in the side image according to the foreground image obtained by the background subtraction method and the human skeleton database. of multiple anchor points.

The image processing device 3 obtains at least two positioning points from the plurality of positioning points of the frontal image and calculates the lengths thereof. For example, the height is calculated according to the positioning points of the human head and the positioning points of the human foot, and the slope of each straight line is calculated according to the polyline of the positioning points of the shoulders and the back of the human body, and the overall connection type is determined.

In practice, the image processing device 3 is, for example, a digital signal processor (digital signal processor), an application specific integrated circuit (ASIC), a digital logic circuit, and a field programmable gate array (FPGA). ) or other hardware components that can perform the above-mentioned image processing functions, which are not limited in the present invention.

In another embodiment of the image recognition system 10 of the present invention, in addition to the first camera device 1 and the second camera device 2 , a third camera device (not shown) is further included. For example, the third camera device is installed on the ceiling of the business hall of a financial institution, and is electrically connected to the image processing device 3 . The third camera device is used for obtaining an overhead image of the user. The image processing device 3 extracts the skin color and black color blocks from the overhead image, calculates the ratio of the two, and determines which type of Norwood's Classification the shape of the black block belongs to.

In yet another embodiment of the image recognition system 10 of the present invention, in addition to the first camera device 1 and the second camera device 2 , a fourth camera device (not shown) is further included. The fourth camera device is electrically connected to the image processing device 3 . The installation position of the fourth camera device 1 can capture the back image of the user standing in front of the designated counter in the business hall of the financial institution. The first camera device 1 and the fourth camera device are depth cameras, so the front image captured by the first camera device 1 further includes the first depth information, and the rear image captured by the fourth camera device further includes the second depth information. The installation positions of the first camera device 1 and the fourth camera device form a first connection line. The standing position of the user and the setting position of the first camera device form a second connection. The standing position of the user and the setting position of the fourth camera device form a third connection line. The first connection line and the second connection line form a first included angle, and the first connection line and the third connection line form a second included angle. In this embodiment, the image processing device 3 is further configured to calculate another posture evaluation of another posture type according to the first depth information, the second depth information, the length of the first connection line, the first included angle and the second included angle value. Specifically, since the length of the first connection line is a fixed value, the image processing device 3 can calculate the distance from the first camera device 1 to the front of the user's body according to the first included angle, the first depth information and basic trigonometric function operations. a distance, and can calculate the second distance between the fourth camera device and the back of the user's body according to the second included angle, the second depth information and basic trigonometric functions, and then deduct the first distance and the second distance from the length of the first connection Two distances can obtain the thickness information of the user's body under the specified positioning point.

The posture database 4 includes a plurality of first table spaces, each of which corresponds to a posture type and includes a plurality of first fields. Described posture types include height, shoulder width, arm length, leg length, hair volume, hair spin, androgenetic stages, hunchback degree, side stance type, body thickness, etc. The present invention does not limit the types and quantities of posture types. .

Each first column of each first table space stores a state assessment value, a weight value and a risk assessment value. The risk assessment value is related to the risk ratio that investors are willing to bear. The weight value is used to determine the proportions of various body types when evaluating by synthesizing multiple body types. For example, the table space corresponding to height is shown in the following table: Height evaluation value (cm) Risk assessment value Weights 141~150 0.32 1 151~160 0.35 1 161~170 0.37 1 171~180 0.39 1.1

The table space corresponding to the side stance is shown in the following table: Side Stance Type Risk assessment value Weights spoon 0.29 1.22 leaning tower 0.37 1.11 bridge 0.43 1.47 flat back 0.53 1.23

In an embodiment of the present invention, the posture assessment values (or posture assessment types), weight values and risk assessment values listed in the above table are obtained through training in advance through a computing device according to a historical database. The historical database includes the front image and profile image retained by the financial institution when the customer opens an account, the investor risk attribute analysis questionnaire filled out by the customer, and the financial product investment records accumulated by the customer in the process of executing business with the financial institution, etc. Big Data. The aforementioned training is specifically performed by the computing device executing a deep learning algorithm and a classifier algorithm. The deep learning algorithm is, for example, one of a recurrent neural network (Recursice Neural Network, RNN) and a Long Short Term Memory (Long Short Term Memory, LSTM). The classifier algorithm is, for example, one of a support vector machine (SVM) and a multi-layer perceptron (MLP). Through the pre-training of the computing device, the posture types corresponding to the normality of various financial products can be obtained, and then the posture assessment value, weight value and risk assessment value stored in each column of each table space can be calculated.

Please refer to Figure 1. The financial product database 5 includes a plurality of second table spaces. Each second table space corresponds to a risk ratio interval and stores information of a financial product. For example, the corresponding risk ratio range of "conservative" financial products is 0~6, which means that investors are basically unwilling to take any investment risk. Therefore, the financial product information recorded in this table space tends to be rewarded from interest income. The product. The corresponding risk ratio range of "prudent" financial products is 7~13, which means that investors can basically accept slight losses in exchange for slight potential investment returns. Therefore, the financial product information recorded in this table space is financial Projects assessed by the agency as low risk. It should be noted that the boundary value of the above risk ratio interval can also be obtained in advance by running the deep learning algorithm and the classifier algorithm through the aforementioned computing device, or defined by the relevant person in charge of the financial institution. be restricted.

Please refer to Figure 1. The computing device 6 is communicatively connected to the image processing device 3 , the posture database 4 and the financial product database 5 .

The computing device 6 obtains one or more risk assessment values from the posture database 4 according to the posture information such as the length, slope and connection type of the two positioning points measured by the image processing device 3, and calculates a comprehensive risk assessment value according to these risk assessment values. , and then select a second table space in the financial product database 5 according to the risk ratio interval corresponding to the comprehensive risk assessment value and output the corresponding financial product information. For example, the computing device 6 may search the posture database 4 to obtain the second risk assessment value and the second weight value according to posture information such as height and side stance, and calculate the comprehensive risk assessment value according to the second risk assessment value and the second weight value. Following the example in the previous table, the risk assessment value corresponding to a height of 165 is 0.37, and the weight value is 1; the risk assessment value corresponding to a user with a straight back is 0.53, and the weight value is 1.23. The computing device 6 adds up these risk assessment values according to their corresponding weight values to obtain a comprehensive risk assessment value of the user (for example, 0.37*1+0.53*1.23=1.0219). The computing device 6 obtains and outputs corresponding financial product information from the financial product database 5 according to the comprehensive risk assessment value.

Based on the above, the image recognition system disclosed in the present invention can obtain the front and side images of the user from a plurality of camera devices installed in the business hall of the financial institution, and obtain the body posture information of the human body in the images through the image processing technology, and Convert multiple body information into risk assessment scores. Therefore, when a customer visits a specific counter in the business hall of a financial institution, the image recognition system disclosed in this case can further recommend financial products suitable for the customer according to the range of the comprehensive risk assessment value, thereby allowing the customer to have a good financial service experience . In addition, the body posture information measured by the image recognition system of the present invention can also be collected for reference by customers.

Although the present invention is disclosed in the foregoing embodiments, it is not intended to limit the present invention. Changes and modifications made without departing from the spirit and scope of the present invention belong to the scope of patent protection of the present invention. For the protection scope defined by the present invention, please refer to the attached patent application scope.

10: Image recognition system 1: The first camera device 2: Second camera device 3: Image processing device 4: Posture database 5: Financial product database 6: Computing device

FIG. 1 is a block diagram of an image recognition system according to an embodiment of the present invention. FIG. 2 is a configuration diagram of a camera device according to an image recognition system according to an embodiment of the present invention.

10: Image recognition system

1: The first camera device

2: Second camera device

3: Image processing device

4: Posture database

5: Financial product database

Claims (6)

An image recognition system with multiple camera devices, comprising: a first camera device for obtaining a frontal image when a user is standing; a second camera device for obtaining a side image when the user is standing; a third camera device for obtaining an overhead image of the user; an image processing device electrically connected to the first camera device, the second camera device and the third camera device, the image processing device for obtaining The front image selects a plurality of positioning points associated with the user and calculates a length of two positioning points among the positioning points, selects a plurality of positioning points associated with the user from the side image and calculates among the positioning points A slope and a connection type of two positioning points, and selecting a plurality of positioning points from the overhead image and calculating a block formed by the positioning points and a color distribution of the block; an integrated database, including complex numbers a first table space, each of the first table spaces corresponds to a posture type and includes a plurality of first fields, each of the first fields stores a posture assessment value and a risk assessment value of the posture type and a weight value, wherein the posture types include: height, shoulder width, arm length, leg length, hunchback degree, side stance type and body thickness; a financial product database, including a plurality of second table spaces, each The second table spaces correspond to a risk ratio range and store a financial product information; a first computing device communicates with the image processing device, the posture database and the financial product database, and the first computing device according to the The length, the slope and the connection type are obtained from the posture database, and a plurality of the risk assessment values are obtained, a comprehensive risk assessment value is calculated according to the obtained risk assessment values, and a comprehensive risk assessment value is calculated from the financial product database selecting one of the second table spaces to output the corresponding financial product information, wherein the comprehensive risk assessment value belongs to the risk ratio range corresponding to the selected second table; and a second computing device, connected to the communication The posture database and the financial product database, the second computing device is used for running a deep learning algorithm and a classifier algorithm according to a historical database to update the posture assessment value and the risk assessment value to the posture data database and update the risk ratio range to the financial product database, wherein the historical database is used to store the front image, the side image and the financial product investment records of each of a plurality of customers. The image recognition system with multiple camera devices as claimed in claim 1, further comprising: a fourth camera device electrically connected to the image processing device, and the fourth camera device is used to obtain a back image of the user standing ; wherein, the first camera device and the fourth camera device are depth cameras, and the setting positions of the first camera device and the fourth camera device form a first connection; the front image further includes a first depth information, The back image further includes a second depth information; the image processing device is further configured to calculate another posture evaluation value of another posture type according to the first depth information, the second depth information, and a length of the first connection . The image recognition system with multiple cameras according to claim 2, wherein the standing position of the user and the setting position of the first camera device form a second connection; the standing position of the user and the fourth camera The installation position of the device forms a third connection; the first connection and the second connection form a first angle; the first connection and the third connection form a second angle; and the image processing device It further includes calculating the other posture evaluation value according to the first included angle and the second included angle. The image recognition system with multiple cameras as claimed in claim 1, wherein the deep learning algorithm is one of a long short-term memory neural network or a recurrent neural network, and the classifier algorithm is a multilayer perceptron or a support vector one of the machines. The image recognition system with multiple cameras as claimed in claim 1, wherein the image processing module is further configured to perform color space conversion, color filtering and edge detection according to the front image and the side image. The image recognition system with multiple cameras as claimed in claim 1, wherein the image processing device is further configured to convert a color space of the front image and the side image into HSV color space, CIE 1931 color space, and YIQ color space and one of the YCbCr color spaces.

Images