TW202123132A - Image recognition system having multiple cameras - Google Patents

Abstract

An image recognition system comprises two camera devices, an image processing device, a posture database, a financial product database, and a computing device. Said two camera devices respectively obtain an image of a front view and an image of a side view. The image processing device selects a plurality of positioning points from the image of the front view and the side view and calculates a length, a slope, and a connection type of two of these positioning points. Each tablespace of the posture database corresponds to a posture type and comprises a plurality of fields. Each field stores a posture value and a risk evaluation value. Each tablespace of the financial product database corresponds to an interval of a risk ratio and stores the information of a financial product. The computing device calculates an integrated risk evaluation value and outputs the information of the financial product.

Description

Image recognition system with multiple camera devices

The present invention relates to an image recognition system, particularly an image recognition system that obtains human body information from images.

For example, financial institutions such as banks and post offices have set up multiple surveillance cameras in their business halls to monitor people moving in and out of the business halls.

However, the surveillance images taken by these surveillance cameras are usually used to review special events that occurred at a specific point in time afterwards (such as masked robbery, driver theft, etc.). In other words, the monitoring system set up by financial institutions lacks a mechanism for real-time processing and reporting based on current images. Although the surveillance system runs for a long time, it is only used as a safety device to deter interested people. Furthermore, the surveillance picture taken from a height must include all kinds of people and objects, such as waiting people, people at the counter, counter clerk, lobby floor, waiting seats, ATMs, discount machines... etc.; It is not easy to arrange a dedicated person to watch the surveillance video, and quickly lock the posture of a specific person from a complex picture, not to mention that humans are prone to lose important information in the picture due to fatigue or distraction.

In addition, for frequently frequent high-net-worth customers, financial institutions provide exclusive financial services for customers in this class, and through customized product investment portfolios, high-net-worth customers can increase their wealth. On the other hand, financial institutions usually assist customers who are generally willing to conduct wealth management to make mid- and long-term asset planning through their extensive marketing bases. By understanding the customers’ current cash flow and future capital use planning, they conduct risk assessments to propose a complete financial system. Product recommendation plan.

However, for customers who simply come to the business hall to handle general financial services, the bank staff or financial specialists of financial institutions cannot know whether the customer is a customer with potential financial needs based on the customer’s posture. Therefore, when the bank staff promotes financial products It is often formalized, unable to provide financial products that may meet the needs of this type of customer, and can only blindly promote projects that customers are not necessarily interested in, thus wasting the time and energy of both parties.

In view of this, the present invention proposes an image recognition system with multiple camera devices, which can obtain the user's full-body image from the images captured by multiple surveillance cameras with different shooting angles, and further calculate the measurable information of the body posture . Using these posture values to analyze in a pre-trained database can increase the sales success rate of financial products and reduce the probability that customers will be unhappy due to unsuitable products being recommended.

An image recognition system with multiple camera devices according to an embodiment of the present invention includes a first camera device, a second camera device, an image processing device, a posture database, a financial product database, and a computing device. The first camera device obtains a frontal image when the user is standing. The second camera device obtains a profile image of the user when he is standing. The image processing device is electrically connected to the first camera device and the second camera device. The image processing device selects a plurality of positioning points associated with the user from the front image, and calculates the length of two positioning points among these positioning points. The image processing device selects a plurality of anchor points associated with the user from the silhouette image, and calculates the slope and connection type of two anchor points among these anchor points. The posture database includes a plurality of first table spaces. Each first table space corresponds to a unified state type. Each first table space includes a plurality of first fields. Each first field stores the posture evaluation value and risk evaluation value of the posture type. The financial product database includes a plurality of second table spaces. Each second table space corresponds to a risk ratio interval and stores a financial product information. The computing device communicates with the image processing device, the posture database and the financial product database. The computing device obtains a plurality of risk assessment values from the body database according to the length, slope, and connection type. The computing device calculates a comprehensive risk assessment value based on the obtained risk assessment values. The computing device selects a second table space from the financial product database to output corresponding financial product information, where the comprehensive risk assessment value belongs to the risk ratio interval corresponding to the selected second table.

With the above architecture, the image recognition system disclosed in this case can obtain the front and side images of the user from multiple camera devices installed in the business halls of financial institutions, and obtain the posture information of the human body in the image through image processing technology, and integrate Multiple posture information is converted into risk assessment scores. Therefore, when a customer visits a specific counter in the business hall of a financial institution, the image recognition system disclosed in this case can further recommend financial products suitable for the customer based on the range of the comprehensive risk assessment value, thereby allowing the customer to have a good financial service experience . In addition, the posture information measured by the image recognition system of the present invention can also be compiled for customers' reference.

The above description of the disclosure and the following description of the embodiments are used to demonstrate and explain the spirit and principle of the present invention, and to provide a further explanation of the scope of the patent application of the present invention.

The detailed features and advantages of the present invention will be described in detail in the following embodiments. The content is sufficient to enable anyone familiar with the relevant art to understand the technical content of the present invention and implement it accordingly, and according to the content disclosed in this specification, the scope of patent application and the drawings. Anyone who is familiar with relevant skills can easily understand the purpose and advantages of the present invention. The following examples further illustrate the viewpoints of the present invention in detail, but do not limit the scope of the present invention by any viewpoint.

Please refer to FIG. 1, which is a block diagram of an image recognition system 10 according to an embodiment of the present invention. The image recognition system 10 can be installed in a business hall of a financial institution, and is suitable for users (customers) who are standing at designated counters and are expected to handle financial-related businesses. The image recognition system 10 includes: a first camera device 1, a second camera device 2, an image processing device 3, a posture database 4, a financial product database 5, and a computing device 6.

Please refer to FIG. 2, which is a schematic top view of the first camera device 1 and the second camera device 2 installed in the business hall of a financial institution. As shown in Figure 2, the installation position of the first camera device 1 can capture the frontal image of the user standing in front of the designated counter in the business hall of the financial institution, and the installation position of the second camera device 2 can be taken in the business hall of the financial institution A silhouette of a user standing in front of a designated counter. In practice, two second camera devices 2 can be provided to respectively capture the left and right images of the user. The present invention does not limit the number of second camera devices 2.

Please refer to Figure 1. The image processing device 3 is electrically connected to the first camera device 1 and the second camera device 2. The image processing device 3 can search for the existence of the user in the environment by using the image information such as the front image and the side image captured by the first camera device 1 and the second camera device 2 and complete the identification of the user's posture. In detail, the image processing device 3 first performs color space conversion, color filtering, and edge detection for the front image and the side image, respectively.

The image processing device 3 can convert the RGB color space of the front image and the side image into one of HSV color space, CIE 1931 color space, YIQ color space and YCbCr color space, thereby reducing the impact of light on subsequent human contour detection. influences.

After the color space conversion is completed, the image processing device 3 extracts image blocks of skin color from the original front image and side image, and filters the blocks of background color at the same time. Specifically, the image processing device 3 defines a specified equation set to represent the skin color distribution after the color space conversion, and another specified equation set to represent the color distribution of the furnishings in the business hall of the financial institution. In practice, the image processing device 3, for example, uses Gaussian Mixture Model (GMM) to establish a description equation of the color distribution of background objects, and then uses background subtraction or adaptive background subtraction. Obtain foreground information, and then perform pixel vertical projection on this foreground information. If the peak of the histogram is within a reasonable range of human height, it means that there may be people at that place. The image processing device 3 uses, for example, Otsu Thresholding (Otsu Thresholding) or Balanced Histogram Thresholding (BHT) to find all pixels that conform to the skin color distribution from the image block where there may be people.

In one embodiment, for the binarized image, the image processing device 3 can use a morphology (dilation) operator and an erosion (erosion) operator to eliminate noise outside the skin color area. Specifically, the image processing device 2 may perform a closing operation (closing operation) of expansion and then corrosion, and then an open operation (corrosion and then expansion), so as to highlight the contour around the skin color block in the image.

After obtaining the images of all skin color blocks, the image processing device 3 uses a connected-component labeling algorithm to find the connected pixels in the skin color block images, and obtains the length and width of each block and coordinate. The image processing device 3 further performs an ellipse comparison for each block, thereby obtaining the skin color block of the human head and the limbs not covered by the clothes. The image processing device 3 uses, for example, one of the Canny filter, the Sobel filter and the Prewitt operator to perform edge detection on the skin color block to obtain the positioning points of the human head and limbs. After obtaining the aforementioned positioning points, the image processing device 3 can calculate a plurality of positioning points associated with the user in the front image and associated with the user in the silhouette image based on the foreground image obtained by the background subtraction method and the human skeleton database. A plurality of anchor points.

The image processing device 3 obtains at least two positioning points from a plurality of positioning points of the front image and calculates the length thereof. For example, the height is calculated based on the positioning points of the human head and the positioning points of the human feet, the slope of each straight line is calculated based on the broken lines of the multiple positioning points of the human shoulder and back, and the overall connection type is judged.

In practice, the image processing device 3 is, for example, a digital signal processor (digital signal processor), an application specific integrated circuit (ASIC), a digital logic circuit, and a field programmable gate array (FPGA). ) Or other hardware components that can perform the above-mentioned image processing functions, which are not limited by the present invention.

In another embodiment of the image recognition system 10 of the present invention, in addition to the first camera device 1 and the second camera device 2, it further includes a third camera device (not shown). The third camera device is, for example, installed on the ceiling of a business hall of a financial institution and is electrically connected to the image processing device 3. The third camera device is used to obtain an overhead image of the user. The image processing device 3 extracts skin color and black color blocks from the overhead image and calculates the ratio of the two, and at the same time determines the shape of the black block belongs to which type of male baldness (Norwood’s Classification).

In another embodiment of the image recognition system 10 of the present invention, in addition to the first camera device 1 and the second camera device 2, it further includes a fourth camera device (not shown). The fourth camera device is electrically connected to the image processing device 3. The setting position of the fourth camera device 1 can capture the back image of the user standing in front of the designated counter in the business hall of the financial institution. The first camera device 1 and the fourth camera device are depth cameras. Therefore, the front image captured by the first camera device 1 further includes the first depth information, and the back image captured by the fourth camera device further includes the second depth information. The installation positions of the first camera device 1 and the fourth camera device form a first connection line. The standing position of the user and the setting position of the first camera device form a second connection. The standing position of the user and the setting position of the fourth camera device form a third connection. The first line and the second line form a first angle, and the first line and the third line form a second angle. In this embodiment, the image processing device 3 is further used to calculate another posture evaluation of another posture type based on the first depth information, the second depth information, the length of the first connection, the first angle and the second angle. value. Specifically, since the length of the first connection is a fixed value, the image processing device 3 can calculate the first angle from the first camera device 1 to the front of the user's body based on the first included angle, the first depth information, and basic trigonometric calculations. The second distance from the fourth camera device to the back of the user’s body can be calculated based on the second included angle, the second depth information, and basic trigonometric calculations. Two distances can obtain the thickness information of the user's body under the specified anchor point.

The posture database 4 includes a plurality of first table spaces, and each first table space corresponds to a posture type and includes a plurality of first fields. The posture types include height, shoulder width, arm length, leg length, hair volume, hair rotation, male baldness staging, degree of hunchback, side stance type, body thickness, etc. The present invention does not limit the type and number of posture types. .

Each first column of each first table space stores an integrated evaluation value, a weight value and a risk evaluation value. The risk assessment value is related to the risk ratio that the investor is willing to bear. The weight value is used to determine the proportion of various posture types when comprehensively evaluating multiple posture types. For example, the table space corresponding to height is shown in the following table: Height assessment value (cm) Risk assessment value Weights 141~150 0.32 1 151~160 0.35 1 161~170 0.37 1 171~180 0.39 1.1

The table space corresponding to the side stance is shown in the following table: Side stance type Risk assessment value Weights Spoon 0.29 1.22 Leaning tower 0.37 1.11 Bridge 0.43 1.47 Flat back 0.53 1.23

In an embodiment of the present invention, the posture evaluation values (or posture evaluation types), weight values, and risk evaluation values listed in the above table are obtained by pre-training based on a historical database through a computing device. The historical database includes the frontal images and silhouette images kept by the financial institution based on the customer's account opening, the investor risk attribute analysis questionnaire filled out by the customer, and the financial product investment record accumulated by the customer during the execution of the business with the financial institution, etc. Big Data. The aforementioned training is specifically that the computing device executes a deep learning algorithm and a classifier algorithm. The deep learning algorithm is, for example, one of Recursice Neural Network (RNN) and Long Short Term Memory (LSTM). The classifier algorithm described is, for example, one of a support vector machine (SVM) and a multilayer perceptron (MLP). Through the pre-training of the computing device, the posture type corresponding to the normality of various financial products can be obtained, and then the posture evaluation value, weight value and risk evaluation value stored in each field of each table space can be calculated.

Please refer to Figure 1. The financial product database 5 includes a plurality of second table spaces. Each second table space corresponds to a risk ratio interval and stores a piece of financial product information. For example, "conservative" financial products correspond to a risk ratio range of 0-6, which means that investors are basically unwilling to take any investment risks. Therefore, the financial product information recorded in this table space is biased towards remuneration from interest income The product. "Prudent" financial products correspond to a risk ratio range of 7 to 13, which means that investors can basically accept a slight loss in exchange for a slight potential investment return. Therefore, the financial product information recorded in this table space is financial Projects rated as low-risk by the agency. It should be noted that the boundary value of the above-mentioned risk ratio interval can also be obtained by running the deep learning algorithm and the classifier algorithm through the aforementioned computing device in advance, or be defined by the relevant person in charge of the financial institution. This is not the case in the present invention. To limit.

Please refer to Figure 1. The computing device 6 is communicatively connected to the image processing device 3, the posture database 4, and the financial product database 5.

The computing device 6 obtains one or more risk assessment values from the body database 4 according to the posture information such as the length, slope, and connection type of the two anchor points measured by the image processing device 3, and calculates a comprehensive risk assessment value based on these risk assessment values , And then select a second table space from the financial product database 5 corresponding to the risk ratio interval corresponding to the comprehensive risk assessment value and output the corresponding financial product information. For example, the computing device 6 can obtain the two-risk assessment value and the two-weight value from the posture database 4 according to posture information such as height and side stance, and calculate the comprehensive risk assessment value based on the two-risk assessment value and the two-weight value. Using the example in the previous table, the risk assessment value corresponding to height 165 is 0.37, and the weight value is 1. The risk assessment value corresponding to the user whose side standing is straight back is 0.53, and the weight value is 1.23. The computing device 6 sums these risk assessment values according to their corresponding weight values to obtain a comprehensive risk assessment value of the user (for example, 0.37*1+0.53*1.23=1.0219). The computing device 6 obtains and outputs corresponding financial product information in the financial product database 5 according to the comprehensive risk assessment value.

In summary, the image recognition system disclosed in the present invention can obtain the front and side images of the user from multiple camera devices installed in the business halls of financial institutions, and obtain the posture information of the human body in the image through image processing technology, and Convert multiple posture information into risk assessment scores. Therefore, when a customer visits a specific counter in the business hall of a financial institution, the image recognition system disclosed in this case can further recommend financial products suitable for the customer based on the range of the comprehensive risk assessment value, thereby allowing the customer to have a good financial service experience . In addition, the posture information measured by the image recognition system of the present invention can also be compiled for customers' reference.

Although the present invention is disclosed in the foregoing embodiments, it is not intended to limit the present invention. All changes and modifications made without departing from the spirit and scope of the present invention fall within the scope of the patent protection of the present invention. For the scope of protection defined by the present invention, please refer to the attached scope of patent application.

10: Image recognition system 1: The first camera 2: The second camera device 3: Image processing device 4: Posture database 5: Financial product database 6: Computing device

FIG. 1 is a block diagram of an image recognition system according to an embodiment of the present invention. FIG. 2 is a configuration diagram of a camera device drawn by an image recognition system according to an embodiment of the present invention.

10: Image recognition system

1: The first camera

2: The second camera device

3: Image processing device

4: Posture database

5: Financial product database

Claims (6)

An image recognition system with multiple camera devices includes: a first camera device for obtaining a frontal image when a user is standing; a second camera device for obtaining a side image when the user is standing; A third camera device for obtaining an overhead image of the user; an image processing device electrically connected to the first camera device, the second camera device and the third camera device, and the image processing device is used for downloading The front image selects a plurality of positioning points associated with the user and calculates a length of two positioning points among the positioning points, and selects a plurality of positioning points associated with the user from the silhouette image and calculates the positioning points A slope of two anchor points and a connection type, and a plurality of anchor points are selected from the overhead image and a block formed by the anchor points and a color distribution of the block are calculated; an integrated database, including plural numbers First table spaces, each of the first table spaces corresponds to a body type and includes a plurality of first fields, and each of the first fields stores a posture evaluation value and a risk evaluation value of the posture type And a weight value, where the posture types include: height, shoulder width, arm length, leg length, degree of hunchback, side stance type, and body thickness; a financial product database, including a plurality of second table spaces, each The second table spaces correspond to a risk ratio interval and store financial product information; a first computing device is communicatively connected to the image processing device, the posture database and the financial product database, and the first computing device is based on the The length, the slope, and the connection type obtain a plurality of the risk assessment values from the body database, calculate a comprehensive risk assessment value based on the obtained risk assessment values, and obtain a comprehensive risk assessment value from the financial product database Selecting one of the second table spaces to output the corresponding financial product information, wherein the comprehensive risk assessment value belongs to the risk ratio interval corresponding to the selected second table; and a second computing device, which is connected in communication The posture database and the financial product database, and the second computing device is used for running a deep learning algorithm and a classifier algorithm based on a historical database to update the posture evaluation value and the risk evaluation value to the posture data And update the risk ratio interval to the financial product database, where the historical database is used to store the frontal image, the silhouette image, and the financial product investment record of each of a plurality of customers. The image recognition system with multiple camera devices according to claim 1, further comprising: a fourth camera device electrically connected to the image processing device, and the fourth camera device is used to obtain a back image when the user is standing Wherein, the first camera device and the fourth camera device are depth cameras, and the first camera device and the fourth camera device are arranged at a position to form a first connection; the front image further includes a first depth information, The back image further includes a second depth information; the image processing device is further used for calculating another posture evaluation value of another posture type according to the first depth information, the second depth information, and a length of the first connection . The image recognition system with multiple cameras according to claim 2, wherein the standing position of the user and the setting position of the first camera form a second connection; the standing position of the user and the fourth camera The installation position of the device forms a third connection; the first connection and the second connection form a first angle; the first connection and the third connection form a second angle; and the image processing device It further includes calculating the other posture evaluation value based on the first included angle and the second included angle. The image recognition system with multiple cameras according to claim 1, wherein the deep learning algorithm is one of a long and short-term memory neural network or a recurrent neural network, and the classifier algorithm is a multilayer perceptron or a support vector Machine one of them. The image recognition system with multiple cameras according to claim 1, wherein the image processing module is further used to perform color space conversion, color filtering, and edge detection according to the front image and the side image. The image recognition system with multiple camera devices according to claim 1, wherein the image processing device is further used to convert each of the front image and the side image into HSV color space, CIE 1931 color space, and YIQ color space And YCbCr color space.

Images