TW201942849A - System and method for detecting trading behavior - Google Patents

System and method for detecting trading behavior Download PDF

Info

Publication number
TW201942849A
TW201942849A TW107111707A TW107111707A TW201942849A TW 201942849 A TW201942849 A TW 201942849A TW 107111707 A TW107111707 A TW 107111707A TW 107111707 A TW107111707 A TW 107111707A TW 201942849 A TW201942849 A TW 201942849A
Authority
TW
Taiwan
Prior art keywords
image
user
module
assessment
message
Prior art date
Application number
TW107111707A
Other languages
Chinese (zh)
Other versions
TWI671701B (en
Inventor
張芸浩
Original Assignee
華南商業銀行股份有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 華南商業銀行股份有限公司 filed Critical 華南商業銀行股份有限公司
Priority to TW107111707A priority Critical patent/TWI671701B/en
Application granted granted Critical
Publication of TWI671701B publication Critical patent/TWI671701B/en
Publication of TW201942849A publication Critical patent/TW201942849A/en

Links

Abstract

A system for detecting trading behavior comprises a first camera device, a second device, an image analysis module, a judging module and a message passing module. The first camera device captures a first image including the entire body of the user in the detection area. The second camera device captures a second image of the face of the user operating an automated device. The image analysis module analyzes the first image and the second image and generates a posture assessment and an emotion evaluation. The judging module generates an interactive policy according to the posture assessment and the emotion evaluation. The message passing module selectively sends an indicative message or an alert message according to the interactive policy.

Description

交易行為偵測系統及其方法Trading behavior detection system and method

本發明係應用動態影像辨識技術於金融領域,特別是關於一種在自動化設備執行交易行為的偵測系統及其方法。The invention applies dynamic image recognition technology to the financial field, and particularly relates to a detection system and method for performing transaction behavior on automated equipment.

自動化設備已普及於一般民眾的日常生活,在便利商店、大賣場、銀行、郵局、農會、捷運站等處,裝設有ATM(自動櫃員機,Automated Teller Machine)、自動繳費機或自動加值機等自動化裝置,因而可減少民眾自行攜帶大量現金的需求,提升交易時的便利性。Automation equipment has been popularized in the daily life of ordinary people. ATMs (Automated Teller Machines), automatic payment machines or automatic payment machines are installed in convenience stores, hypermarkets, banks, post offices, farmers' associations, and MRT stations. Automated devices such as check-in can reduce the need for people to carry large amounts of cash on their own and improve convenience during transactions.

雖然自動化設備設置的本意係為減少相關人員的服務時間,然而換個角度而言,即代表自動化設備在大部分時間皆處於無人值守狀態。以ATM為例,現今仍有大多數ATM採用提款卡與密碼對使用者進行認證,認證通過後方能獲得帳戶存取權。但對於ATM使用者在操作ATM時所遭遇的各種違法行為,例如:民眾於銀行ATM提款時被歹徒從背後搶劫,或是詐騙集團引導被害人利用ATM取款或轉帳,抑或是車手在取得受害者提款卡和密碼後進行盜領等諸如此類的不法交易行為,以現行的ATM系統而言,皆無法在第一時間獲得相關人員的即時協助,而僅能依靠ATM周邊設置的監視系統在事後進行補救措施。因此,若能由自動化設備在偵測到可疑行為時主動預警,將提升民眾使用自動化設備時的安全性,增強對自動化設備設置機構的信賴感。Although the original intention of setting up the automation equipment is to reduce the service time of the relevant personnel, from another perspective, it means that the automation equipment is in an unattended state most of the time. Taking ATM as an example, most ATMs still use ATM cards and passwords to authenticate users. After passing the authentication, they can obtain account access. But for the various illegal behaviors encountered by ATM users when operating ATMs, such as: the people were robbed from behind by criminals when withdrawing ATMs from banks, or fraud groups led victims to use ATMs to withdraw or transfer money, or drivers were obtaining victims Illegal transactions such as theft of credit cards and passwords, etc., cannot be immediately obtained by the relevant personnel in the current ATM system, and can only be performed after the fact by relying on the monitoring system installed around the ATM. Remedy. Therefore, if the automated equipment can take the initiative to warn when it detects suspicious behavior, it will improve the safety of the public when using the automated equipment and increase the trust in the automated equipment setting mechanism.

有鑑於此,本發明提出一種交易行為偵測系統以及應用此系統的交易行為偵測方法,藉以解決前述自動化設備無法主動偵測可疑交易行為並主動發出警示的問題。In view of this, the present invention proposes a transaction behavior detection system and a transaction behavior detection method using the system, so as to solve the problem that the aforementioned automated equipment cannot actively detect suspicious transaction behavior and actively issue a warning.

依據本發明一實施例的交易行為偵測系統,用於偵測操作自動化設備之使用者,自動化設備裝設於一偵測區域,所述的系統包括:第一攝像裝置、第二攝像裝置、影像分析模組、判斷模組以及訊息遞送模組。第一攝像裝置取得偵測區域的第一影像,且第一影像中包含使用者之全身。第二攝像裝置取得操作自動化設備之使用者的第二影像,且第二影像中包含使用者之臉部正面。影像分析模組電性連接至第一攝像裝置及第二攝像裝置以接收第一影像及第二影像,影像分析模組分析第一影像並產生體態評估,分析第二影像並產生情緒評估。判斷模組電性連接至影像分析模組,並根據體態評估及情緒評估綜合判斷以產生互動策略。訊息遞送模組電性連接判斷模組,訊息遞送模組根據互動策略選擇性地發出提示訊息或警示訊息。According to an embodiment of the present invention, a transaction behavior detection system is used to detect a user who operates an automated device. The automated device is installed in a detection area. The system includes: a first camera device, a second camera device, Image analysis module, judgment module and message delivery module. The first camera obtains a first image of the detection area, and the first image includes the entire body of the user. The second imaging device obtains a second image of a user operating the automation equipment, and the second image includes the front of the user's face. The image analysis module is electrically connected to the first camera device and the second camera device to receive the first image and the second image. The image analysis module analyzes the first image and generates a posture evaluation, and analyzes the second image and generates an emotional evaluation. The judgment module is electrically connected to the image analysis module, and comprehensively judges according to the posture assessment and the emotion assessment to generate an interactive strategy. The message delivery module is electrically connected to the judgment module, and the message delivery module selectively issues a prompt message or a warning message according to the interaction strategy.

依據本發明另一實施例的交易行為偵測系統,除上述元件外,更包括記錄模組電性連接至影像分析模組及判斷模組,記錄模組收集影像分析模組針對同一使用者所產生之體態評估及情緒評估,且判斷模組更根據記錄模組中關聯於同一使用者之一歷史評估記錄產生互動策略。According to a transaction behavior detection system according to another embodiment of the present invention, in addition to the above components, the recording module further includes a recording module electrically connected to the image analysis module and the judgment module. The recording module collects the image analysis module for the same user The generated posture assessment and emotional assessment, and the judgment module generates an interactive strategy based on the historical assessment records associated with the same user in the recording module.

依據本發明一實施例的交易行為偵測方法,用於偵測操作自動化設備之使用者,包括:取得偵測區域的第一影像,其中偵測區域中裝設有自動化設備,且第一影像中包含使用者之全身;取得操作自動化設備之使用者的第二影像,且第二影像中包含使用者之臉部正面;分析第一影像並產生體態評估;分析第二影像並產生情緒評估;根據體態評估及情緒評估綜合判斷並產生互動策略;以及根據互動策略選擇性地發出提示訊息或警示訊息。According to an embodiment of the present invention, a transaction behavior detection method for detecting a user operating an automated device includes: obtaining a first image of a detection area, wherein the detection area is provided with an automatic device, and the first image The second image of the user operating the automated equipment is obtained, and the second image includes the front of the user's face; the first image is analyzed to generate a posture assessment; the second image is analyzed to generate an emotional assessment; Comprehensively judge and generate interactive strategies based on posture assessment and emotional assessment; and optionally issue prompt messages or warning messages based on interactive strategies.

基於前述揭露內容,本發明提出的一種交易行為偵測系統及其方法,透過兩個攝影裝置分別拍攝使用者全身與臉部正面的影像,並分別針對全身影像及臉部影像各自分析使用者體型與情緒的評估,此一分析過程可參考同一使用者先前留存的影像記錄,而更為精確地判斷使用者的身分以及使用者是否處於正常交易的狀態,同時也可偵測使用者在進行交易時周圍是否出現可疑人員,並且在本發明揭露的交易行為偵測系統發現異常狀況時,視狀況異常程度選擇性地在自動化設備的螢幕顯示訊息告知使用者,或是主動地發送警示訊息給警調機構或金融機構,以便相關單位即時派遣相關人員前來處理,減少使用者的損失。Based on the foregoing disclosure, a transaction behavior detection system and method provided by the present invention respectively capture images of the user's entire body and the front of the face through two photographing devices, and analyze the user's body shape separately for the whole-body image and the facial image. And emotional evaluation, this analysis process can refer to the previous image records of the same user to more accurately determine the identity of the user and whether the user is in a normal transaction state, and can also detect that the user is conducting a transaction Whether there are suspicious people around the time, and when the transaction behavior detection system disclosed by the present invention finds an abnormal condition, selectively displays a message on the screen of the automated device to inform the user, or actively sends a warning message to the alarm Relocate institutions or financial institutions so that relevant units can immediately dispatch relevant personnel to deal with them, reducing user losses.

以上之關於本揭露內容之說明及以下之實施方式之說明係用以示範與解釋本發明之精神與原理,並且提供本發明之專利申請範圍更進一步之解釋。The above description of the contents of this disclosure and the description of the following embodiments are used to demonstrate and explain the spirit and principle of the present invention, and provide a further explanation of the scope of the patent application of the present invention.

以下在實施方式中詳細敘述本發明之詳細特徵以及優點,其內容足以使任何熟習相關技藝者了解本發明之技術內容並據以實施,且根據本說明書所揭露之內容、申請專利範圍及圖式,任何熟習相關技藝者可輕易地理解本發明相關之目的及優點。以下之實施例係進一步詳細說明本發明之觀點,但非以任何觀點限制本發明之範疇。The detailed features and advantages of the present invention are described in detail in the following embodiments. The content is sufficient for any person skilled in the art to understand and implement the technical contents of the present invention. Anyone skilled in the relevant art can easily understand the related objects and advantages of the present invention. The following examples further illustrate the viewpoints of the present invention in detail, but do not limit the scope of the present invention in any way.

本發明所揭露的交易行為偵測系統及其方法係適用於自動化設備與操作此自動化設備的使用者。實務上,所述的自動化設備例如係自動櫃員機、自動繳費機、或自動加值機等硬體。The transaction behavior detection system and method disclosed in the present invention are applicable to an automated device and a user operating the automated device. In practice, the automated equipment is, for example, hardware such as an automatic teller machine, an automatic payment machine, or an automatic value-added machine.

請參考圖1,其係繪示依據本發明一實施例所敘述的交易行為偵測系統的方塊示意圖,包括第一攝像裝置1、第二攝像裝置2、影像分析模組3、判斷模組4、記錄模組5以及訊息遞送模組6。第一攝像模組1及第二攝像模組2各自電性連接至影像分析模組3,判斷模組4電性連接至影像分析模組3及訊息遞送模組6,記錄模組5電性連接至影像分析模組3及判斷模組4。Please refer to FIG. 1, which is a block diagram illustrating a transaction behavior detection system according to an embodiment of the present invention, including a first camera device 1, a second camera device 2, an image analysis module 3, and a judgment module 4. , Recording module 5 and message delivery module 6. The first camera module 1 and the second camera module 2 are each electrically connected to the image analysis module 3, the judgment module 4 is electrically connected to the image analysis module 3 and the message delivery module 6, and the recording module 5 is electrically Connected to the image analysis module 3 and the judgment module 4.

請再參考圖2,其係繪示第一攝像裝置1、第二攝像裝置2及自動化設備AE的位置關係示意圖。自動化設備AE裝設於一偵測區域DZ。Please refer to FIG. 2 again, which is a schematic diagram illustrating a position relationship between the first camera device 1, the second camera device 2, and the automation equipment AE. The automation equipment AE is installed in a detection area DZ.

實務上,第一攝像裝置1及第二攝像裝置2例如係以互補式金屬氧化物半導體(Complementary Metal-Oxide-Semiconductor, CMOS)或感光耦合元件(Charge-Coupled Device, CCD)作為感光元件的標準型監視攝影機或紅外線監視攝影機,本發明並不限制第一攝像裝置1及第二攝像裝置2的硬體種類。In practice, the first imaging device 1 and the second imaging device 2 use, for example, a complementary metal-oxide semiconductor (CMOS) or a photosensitive-coupled device (CCD) as the standard of the photosensitive element Type surveillance cameras or infrared surveillance cameras, the present invention does not limit the types of hardware of the first imaging device 1 and the second imaging device 2.

第一攝像裝置1配置一廣角鏡頭、超廣角鏡頭或魚眼鏡頭。實務上,第一攝像裝置1較佳架設於自動化設備AE的上方以便完整拍攝自動化設備AE及其周圍的偵測區域DZ。必須特別強調的是:第一攝像裝置1所拍攝的第一影像中需包含使用者之全身。The first camera 1 is configured with a wide-angle lens, an ultra-wide-angle lens, or a fisheye lens. In practice, the first camera device 1 is preferably set above the automation equipment AE so as to completely capture the automation equipment AE and the detection area DZ around it. It must be particularly emphasized that the first image captured by the first camera 1 needs to include the entire body of the user.

請繼續參考圖2。第二攝像裝置2用於取得操作自動化設備AE的使用者的臉部正面影像作為第二影像。在本發明其他實施例中,第二攝像裝置2可結合立體攝影技術,使得第二攝像裝置2所拍攝的第二影像包括人類臉部的三維深度資訊。Please continue to refer to Figure 2. The second camera 2 is configured to obtain a front image of the face of a user who operates the automation equipment AE as a second image. In other embodiments of the present invention, the second camera device 2 may be combined with stereo photography technology, so that the second image captured by the second camera device 2 includes three-dimensional depth information of a human face.

在本發明一實施例中,更包括第一感測器電性連接至第一攝像裝置1以及第二感測器電性連接至第二攝像裝置2。第一感測器例如係熱電型紅外線感測器或量子型紅外線感測器,裝設於偵測區域DZ的入口處,藉此偵測使用者是否已進入偵測區域DZ。當第一感測器測得使用者進入偵測區域DZ時,啟動第一攝像裝置1開始取得第一影像。第二感測器用以偵測使用者是否操作自動化設備AE。例如:當自動化設備AE為自動櫃員機時,第二感測器感測使用者是否將金融卡插入讀卡機。當第二感測器測得自動化設備AE正在被操作時,啟動第二攝像裝置2開始取得第二影像。藉由第一感測器和第二感測器的設置,可減少第一攝像裝置1和第二攝像裝置2在自動化設備AE未被使用時持續拍攝導致不必要的電力消耗。In an embodiment of the present invention, the method further includes a first sensor electrically connected to the first camera device 1 and a second sensor electrically connected to the second camera device 2. The first sensor is, for example, a pyroelectric infrared sensor or a quantum infrared sensor, and is installed at the entrance of the detection area DZ to detect whether the user has entered the detection area DZ. When the first sensor detects that the user enters the detection area DZ, the first camera 1 is activated to start acquiring a first image. The second sensor is used to detect whether the user operates the automated equipment AE. For example, when the automatic equipment AE is an ATM, the second sensor detects whether the user inserts a financial card into the card reader. When the second sensor detects that the automated equipment AE is being operated, the second camera device 2 is activated to start acquiring a second image. With the arrangement of the first sensor and the second sensor, it is possible to reduce unnecessary power consumption caused by the continuous shooting of the first camera device 1 and the second camera device 2 when the automation equipment AE is not in use.

請參考圖1。影像分析模組3接收第一影像及第二影像。實務上,影像分析模組3例如係一般個人電腦之中央處理器(Central Processing Unit,CPU)、微處理器(Microprocessor)、數位信號處理器(Digital Signal Processor,DSP)、特殊應用積體電路(Application-Specific Integrated Circuit,ASIC)、其他類似元件或上述元件之組合, 本發明並不特別限制影像分析模組3的硬體類型。Please refer to Figure 1. The image analysis module 3 receives a first image and a second image. In practice, the image analysis module 3 is, for example, a central processing unit (CPU), a microprocessor (microprocessor), a digital signal processor (DSP), or a special application integrated circuit ( Application-Specific Integrated Circuit (ASIC), other similar components, or a combination of the above components, the present invention does not specifically limit the hardware type of the image analysis module 3.

影像分析模組3執行影像處理演算法分析第一影像及第二影像,並分別輸出體態評估以及情緒評估。體態評估包括使用者的體型及走路姿態,具體而言例如透過畫面中移動物件與背景物件的比例計算使用者的體型大小。此外,更可透過深度學習的方式讓影像分析模組3判斷在偵測區域DZ中的複數個人員,並區分這些人員是否正在使用自動化設備AE。影像分析模組3亦可結合物件資料庫,藉此辨識出第一影像中使用者攜帶或配戴的物品(例如:口罩、安全帽或棍棒等)來作為使用者行為判斷的依據。承上所述,分析第一影像以產生體態評估的演算法可根據實際需求而增加或減少不同組合的判斷規則。The image analysis module 3 executes an image processing algorithm to analyze the first image and the second image, and outputs a posture assessment and an emotion assessment, respectively. The body shape evaluation includes the user's body shape and walking posture. Specifically, for example, the user's body size is calculated from the ratio of moving objects to background objects in the screen. In addition, the image analysis module 3 can also be used to determine the plurality of persons in the detection area DZ through a deep learning method, and distinguish whether these persons are using the automated equipment AE. The image analysis module 3 can also be combined with an object database to identify items (such as masks, helmets, or sticks) carried or worn by the user in the first image as a basis for judging user behavior. As mentioned above, the algorithm that analyzes the first image to generate the posture assessment may increase or decrease the judgment rules of different combinations according to actual needs.

情緒評估包括一情緒類型。詳言之,影像分析模組3在接收第二影像後,從影像中辨識取得使用者臉部輪廓,並擷取臉部的多個特徵點進行運算,藉此判斷使用者臉部正面影像所代表的微表情(Microexpression),並輸出一情緒類型。研究指出,人類的微表情無法掩飾,即使嘗試掩飾,也只能在表情發生後才能進行掩飾,故其有判斷之價值。在本發明一實施例中,影像分析模組3所輸出的情緒類型屬於開心、不屑、憤怒、恐懼、驚訝、憎惡及悲傷其中一者或數者。進一步地,影像分析模組3更包括分析第二影像在第一時間點及第二時間點之臉部特徵點變化量,且情緒評估更包括根據臉部特徵點變化量計算的一表情變化指數。藉此影像分析模組3能更精確地評估使用者的表情變化,供判斷模組4作進一步地評估。Emotional assessment includes an emotional type. In detail, after receiving the second image, the image analysis module 3 recognizes and obtains the contour of the user's face from the image, and extracts multiple feature points of the face for calculation, thereby determining the image of the front of the user's face. Microexpression, and output an emotion type. Studies have pointed out that human micro-expressions cannot be concealed, and even if they are tried to conceal, they can only be concealed after the expressions have occurred, so they have the value of judgment. In one embodiment of the present invention, the types of emotions output by the image analysis module 3 belong to one or more of happiness, disdain, anger, fear, surprise, hatred, and sadness. Further, the image analysis module 3 further includes analyzing a facial feature point change amount of the second image at the first time point and the second time point, and the emotion evaluation further includes an expression change index calculated according to the facial feature point change amount. . In this way, the image analysis module 3 can more accurately evaluate the user's facial expression changes for the judgment module 4 to further evaluate.

實務上,影像分析模組3在分析完成後,將體態評估及情緒評估一併發送至判斷模組4,同時亦儲存至記錄模組5。記錄模組5分別收集每個使用者在操作自動化設備AE時各自的體態評估及情緒評估資料。記錄模組5實質可為內建儲存媒體或外接式儲存媒體,例如:記憶卡或硬碟。判斷模組4接收影像分析模組3發送過來的體態評估及情緒評估,同時根據記錄模組5中關聯於使用者之一歷史評估記錄綜合判斷以產生一互動策略。訊息遞送模組6根據互動策略選擇性地發出提示訊息或警示訊息。提示訊息係透過自動化設備AE的螢幕所顯示的文字訊息,警示訊息係手機簡訊,或保全系統的警報訊號。In practice, after the analysis is completed, the image analysis module 3 sends the posture assessment and the emotion assessment to the judgment module 4 together, and also stores them to the recording module 5. The recording module 5 separately collects each body's physical evaluation and emotional evaluation data when operating the automated equipment AE. The recording module 5 may be a built-in storage medium or an external storage medium, such as a memory card or a hard disk. The judgment module 4 receives the posture assessment and the emotion assessment sent from the image analysis module 3, and simultaneously makes a comprehensive judgment based on a historical assessment record associated with the user in the recording module 5 to generate an interactive strategy. The message delivery module 6 selectively issues a prompt message or a warning message according to the interaction strategy. The alert message is a text message displayed on the screen of the automatic equipment AE, and the alert message is a mobile phone message or an alarm signal of the security system.

關於判斷模組4進行綜合判斷的部分,試舉數例說明如下:使用者至自動櫃員機存款,根據該金融卡對應的帳號,判斷模組4在記錄模組5中查詢此帳號過去累計的歷史體態評估記錄及歷史情緒評估記錄,各自比較本次由影像分析模組3送來的體態評估及情緒評估後,發現使用者的體型差異與其臉部特徵點變化量兩者的差異程度其中一者超過各自的合理閾值,因此判斷模組4認定本次操作非帳戶所有人,並且透過訊息遞送模組6發送警示訊息通知後勤人員前往自動櫃員機進一步了解情況。進一步地,判斷模組4除了計算體態評估差異值與情緒評估各自的差異值,更包括計算此兩差異值的差異程度,若此差異程度超過一預設閾值,則即使體態評估差異值與情緒評估各自的差異值皆未超過各自的合理閾值,判斷模組4同樣透過訊息遞送模組6發送警示訊息。以實際數據舉例來說,交易行為偵測系統預先設置體態評估的合理閾值為30%,情緒評估的合理閾值為20%,預設閾值為10%。若本次判斷模組4計算出體態評估差異值為29%,情緒評估差異值為5%,則因兩者差異值為24%,超過預設閾值10%,即使體態評估差異值與情緒評估差異值皆在各自的合理閾值之內,判斷模組4仍是透過訊息遞送模組6發出警示訊息。藉由上述判斷模組4的綜合判斷機制,可降低系統未偵測到潛在具有風險的交易行為的機率,並且進一步提升本發明一實施例所敘述的交易行為偵測系統的安全性。Regarding the comprehensive judgment part of the judgment module 4, a few examples are described as follows: The user deposits to the ATM, and according to the account number corresponding to the financial card, the judgment module 4 queries the record module 5 for the accumulated history of the account in the past. Posture evaluation records and historical emotional evaluation records. After comparing the postural evaluation and sentiment evaluation sent by the image analysis module 3, we found one of the differences between the user's body shape and the change in facial feature points. Exceed their respective reasonable thresholds, so the judgment module 4 determines that this operation is not the account owner, and sends a warning message through the message delivery module 6 to notify the logistics personnel to go to the ATM to learn more about the situation. Further, in addition to calculating the difference between the posture assessment difference value and the emotion assessment, the judgment module 4 further includes calculating the difference degree between the two difference values. If the difference degree exceeds a preset threshold, even if the difference between the posture assessment and the emotion is It is evaluated that each of the difference values does not exceed the respective reasonable threshold value, and the judgment module 4 also sends a warning message through the message delivery module 6. Taking actual data as an example, the transaction behavior detection system presets a reasonable threshold value for posture assessment to 30%, a reasonable threshold value for emotion assessment to 20%, and a preset threshold value to 10%. If this judgment module 4 calculates that the difference in body posture evaluation is 29% and the difference in emotion evaluation is 5%, the difference between the two is 24%, which exceeds the preset threshold of 10%, even if the difference between body posture and emotional evaluation The difference values are all within their respective reasonable thresholds. The judgment module 4 still sends a warning message through the message delivery module 6. The comprehensive judgment mechanism of the judgment module 4 can reduce the probability that the system does not detect a potentially risky transaction behavior, and further improve the security of the transaction behavior detection system described in an embodiment of the present invention.

另一由判斷模組4綜合評估的實際案例說明如下:使用者前往自動化設備AE進行交易,在等待交易完成的空檔時間,自動化設備AE的螢幕播放預設的廣告訊息。影像分析模組3在分析第二影像時,將第一時間點設定為廣告播放開始前,將第二時間點設定為廣告播放結束後,且影像分析模組3提供的情緒評估中包括從使用者臉部特徵點的變化量計算出的表情變化指數。據此,判斷模組4可從此表情變化指數判斷廣告訊息對於使用者的影響力屬於正面或是負面,並且可修改下一次互動策略所設定的廣告內容。Another practical case of comprehensive evaluation by the judgment module 4 is as follows: The user goes to the automated equipment AE to conduct a transaction, and during the idle time waiting for the transaction to be completed, the screen of the automated equipment AE plays a preset advertisement message. When the image analysis module 3 analyzes the second image, the first time point is set to before the advertisement playback starts, and the second time point is set to after the advertisement playback ends, and the emotional evaluation provided by the image analysis module 3 includes the use of The expression change index calculated from the change amount of the person's facial feature points. Based on this, the judgment module 4 can judge whether the influence of the advertisement message on the user is positive or negative from the expression change index, and can modify the advertisement content set by the next interaction strategy.

另一由判斷模組4綜合評估的實際案例說明如下:自動化設備AE調整使用介面,操作步驟或是增加新功能,在使用者操作新版介面時,判斷模組4根據影像分析模組3提供的情緒評估中判斷使用者在操作自動化設備時,是屬於開心的情緒或是屬於悲傷或憎惡的情緒。據此,判斷模組4可判斷使用者對於自動化設備的介面更新是否滿意。Another practical case of comprehensive evaluation by the judgment module 4 is explained as follows: automatic equipment AE adjusts the use interface, operation steps or adds new functions. When the user operates the new interface, the judgment module 4 is based on the analysis provided by the image analysis module 3. In the emotional evaluation, it is determined whether the user belongs to a happy mood or a sad or disgusted emotion when operating an automated device. According to this, the determination module 4 can determine whether the user is satisfied with the interface update of the automation equipment.

另一由判斷模組4進行綜合判斷的實際案例說明如下:判斷模組4從第一影像中發現有兩名人員先後進入偵測區域DZ中,並且從兩人站定的位置及臉部方向判斷出其中一人正在觀察前方操作自動櫃員機的使用者,判斷模組4再從第二影像的情緒評估中發現操作自動櫃員機的使用者的恐懼情緒超過一合理閾值,因此判斷模組4可驅使訊息遞送模組6透過自動櫃員機的螢幕顯示訊息詢問使用者是否需要進一步的協助,並且當使用者選擇「需要協助」時,自動連線至櫃檯人員或是警察機關以便即時提供協助。Another practical case for comprehensive judgment by the judgment module 4 is described as follows: The judgment module 4 found that two persons have entered the detection area DZ from the first image, and from the position and face orientation of the two persons It is determined that one of them is observing the user operating the ATM in front, and the judgment module 4 finds from the second image's emotional evaluation that the fear of the user who operates the ATM exceeds a reasonable threshold, so the judgment module 4 can drive the message The delivery module 6 asks the user through the on-screen display of the ATM whether the user needs further assistance, and when the user selects "Need Assistance", it automatically connects to the counter staff or the police to provide immediate assistance.

又一由判斷模組4進行綜合判斷的實際案例說明如下:使用者至自動櫃員機存款,判斷模組4從第二影像的情緒評估中判斷使用者處於開心情緒,因此判斷模組4可驅使訊息遞送模組6透過自動櫃員機的螢幕顯示訊息,詢問使用者是否對於某近期最高報酬的投資基金有興趣,並且當使用者選擇「進一步了解」時,訊息遞送模組6透過螢幕顯示行銷訊息。Another practical case of comprehensive judgment by the judgment module 4 is explained as follows: the user deposits to the ATM, the judgment module 4 judges that the user is in a happy mood from the emotional evaluation of the second image, so the judgment module 4 can drive the message The delivery module 6 displays a message on the screen of the ATM to ask the user if he is interested in a certain investment fund with the highest return in the near future, and when the user selects "more information", the message delivery module 6 displays a marketing message on the screen.

請參考圖3,其係繪示依據本發明一實施例所敘述的一種交易行為偵測方法,包括下列步驟:Please refer to FIG. 3, which illustrates a method for detecting transaction behavior according to an embodiment of the present invention, including the following steps:

步驟S0:第一感測器感測到使用者接近自動化設備AE後,發出第一感測訊號。Step S0: After the first sensor detects that the user approaches the automated equipment AE, it sends a first sensing signal.

步驟S1:第一攝像裝置1收到第一感測訊號後,開始拍攝偵測區域DZ中包括使用者全身的第一影像。Step S1: After receiving the first sensing signal, the first camera device 1 starts to capture a first image of the entire body of the user in the detection area DZ.

步驟S2:第二感測器感測到使用者操作自動化設備AE後,發出第二感測訊號。Step S2: After the second sensor detects that the user operates the automation equipment AE, it sends a second sensing signal.

步驟S3:第二攝像裝置2收到第二感測訊號後,開始拍攝操作自動化設備AE之使用者的臉部正面作為第二影像。Step S3: After receiving the second sensing signal, the second camera device 2 starts shooting the front of the face of the user who operates the automated equipment AE as the second image.

步驟S4:影像分析模組3分析第一影像並產生體態評估,體態評估包括使用者的體型及走路姿態Step S4: The image analysis module 3 analyzes the first image and generates a posture assessment, which includes the user's body shape and walking posture

步驟S5:影像分析模組3分析第二影像並產生情緒評估,情緒評估更包括第二影像在第一時間點及第二時間點之臉部特徵點變化量,以及根據臉部特徵點變化量計算的一表情變化指數。Step S5: The image analysis module 3 analyzes the second image and generates an emotional evaluation. The emotional evaluation further includes a change amount of the facial feature points of the second image at the first time point and the second time point, and a change amount according to the facial feature points. A calculated expression change index.

步驟S6:判斷模組4根據體態評估及情緒評估綜合判斷並產生互動策略。詳言之,在本發明一實施例中,判斷模組4比對影像分析模組3發送過來的體態評估與記錄模組5中儲存的使用者歷史體態評估記錄以得到一體態評估差異值,以及判斷模組4比對影像分析模組3發送過來的情緒評估與記錄模組5中儲存的使用者歷史情緒評估記錄以得到一情緒評估差異值;當體態評估差異值與情緒評估差異值兩者之差異大於一預設閾值時,則判斷模組4將互動策略設置為「發送警示」。舉例來說,交易行為偵測系統預先設置體態評估差異值與情緒評估差異值之差異的合理閾值為10%。因此,若本次判斷模組4測得的體態評估差異值為29%,情緒評估差異值為5%,則因體態評估差異值與情緒評估差異值之差異24%大於合理閾值10%,則必須將互動策略設置為「發送警示」。透過本實施例步驟S6所述的綜合判斷機制,本交易行為偵測方法可大幅降低違法之徒透過化妝、模仿體態或是使用面具等手段,同時通過體態評估及情緒評估但其中一者為勉強通過的機率,因此可大幅提高交易行為的安全性。Step S6: The judgment module 4 comprehensively judges and generates an interactive strategy based on the posture assessment and the emotion assessment. In detail, in an embodiment of the present invention, the judgment module 4 compares the posture evaluation sent by the image analysis module 3 with the user historical posture evaluation record stored in the recording module 5 to obtain the integrated evaluation difference value. And the judgment module 4 compares the sentiment evaluation sent by the image analysis module 3 with the historical user sentiment evaluation record stored in the recording module 5 to obtain a sentiment evaluation difference value; When the difference is greater than a preset threshold, the judgment module 4 sets the interaction strategy to "send alert". For example, the transaction behavior detection system presets a reasonable threshold for the difference between the difference between the posture assessment and the difference between the emotional assessments as 10%. Therefore, if the difference between the posture assessment difference measured by the judgment module 4 is 29% and the emotion assessment difference is 5%, then the difference between the posture assessment difference and the emotion assessment difference of 24% is greater than the reasonable threshold of 10%. The interaction strategy must be set to Send Alert. Through the comprehensive judgment mechanism described in step S6 of this embodiment, this transaction behavior detection method can greatly reduce the number of offenders who make up, imitate their posture, or use masks. They also pass their posture evaluation and emotional evaluation, but one of them is reluctant. The probability of passing can therefore greatly improve the security of trading behavior.

另一方面,若判斷模組4在上述關於體態評估差異值及情緒評估差異值的計算與比對之後並未發現異狀,則判斷模組4可根據情緒評估中的情緒類型決定互動策略為發送金融產品行銷資訊或是發送其他廣告資訊。On the other hand, if the judgment module 4 does not find any abnormalities after the calculation and comparison of the difference in posture assessment and the difference in emotion assessment, the judgment module 4 may determine the interaction strategy according to the type of emotion in the emotion assessment as Send financial product marketing information or other advertising information.

請參考步驟S7: 訊息遞送模組6根據互動策略發出提示訊息或警示訊息。具體而言,訊息遞送模組6可透過自動化設備的螢幕發出提示訊息,警示訊息則係透過與警調機關的通訊連線發送,或是透過手機簡訊方式發送至使用者指定的號碼。Please refer to step S7: The message delivery module 6 sends a prompt message or a warning message according to the interaction strategy. Specifically, the message delivery module 6 can send a prompt message through the screen of an automated device, and the warning message is sent through a communication connection with the police department or through a mobile phone text message to a user-specified number.

綜合以上所述,本發明之交易行為偵測系統,採用遠距離廣角鏡頭及近距離面部鏡頭的的搭配,以提高行為辨識的準確性,同時透過記錄模組收集以累積客戶的影像資訊,建立個人體態模型及情緒評估的歷史記錄來學習客戶慣性行為模式。以上述資訊為基礎,判斷模組可即時根據使用者當前的體態評估與情緒評估,判斷出是否有危害到使用者進行交易行為的狀況,並且透過自動化設備的螢幕向使用者發送互動式詢問訊息,或逕自發送警示訊息通知相關人員。訊息遞送模組亦可以在適當時機推銷銀行商品,藉此自動化設備可以和使用者具有更多互動方式,並且兼具主動預警的功效。To sum up, the transaction behavior detection system of the present invention uses a combination of a long-range wide-angle lens and a short-range facial lens to improve the accuracy of behavior recognition, and at the same time, collects image information of customers through a recording module to create an individual History of posture models and emotional assessments to learn customer inertial behavior patterns. Based on the above information, the judging module can instantly determine whether the user is in a situation that is harmful to the transaction behavior based on the user's current physical and emotional assessment, and send interactive inquiry messages to the user through the screen of the automated device , Or simply send a warning message to notify the person concerned. The message delivery module can also promote bank merchandise at the right time, so that the automated equipment can have more interactive ways with the user, and also has the effect of active warning.

雖然本發明以前述之實施例揭露如上,然其並非用以限定本發明。在不脫離本發明之精神和範圍內,所為之更動與潤飾,均屬本發明之專利保護範圍。關於本發明所界定之保護範圍請參考所附之申請專利範圍。Although the present invention is disclosed in the foregoing embodiments, it is not intended to limit the present invention. Changes and modifications made without departing from the spirit and scope of the present invention belong to the patent protection scope of the present invention. For the protection scope defined by the present invention, please refer to the attached patent application scope.

AE‧‧‧自動化設備AE‧‧‧Automation equipment

DZ‧‧‧偵測區域DZ‧‧‧ Detection area

1‧‧‧第一攝像裝置1‧‧‧ the first camera

2‧‧‧第二攝像裝置2‧‧‧Second camera device

3‧‧‧影像分析模組3‧‧‧Image Analysis Module

4‧‧‧判斷模組4‧‧‧ Judgment Module

5‧‧‧記錄模組5‧‧‧Recording module

6‧‧‧訊息遞送模組6‧‧‧Message Delivery Module

S0~S7‧‧‧步驟S0 ~ S7‧‧‧step

圖1係依據本發明一實施例所敘述的交易行為偵測系統的方塊示意圖。 圖2係第一攝像裝置、第二攝像裝置及自動化設備的位置關係示意圖。 圖3係依據本發明一實施例所繪示的交易行為偵測方法流程圖。FIG. 1 is a block diagram of a transaction behavior detection system according to an embodiment of the present invention. FIG. 2 is a schematic diagram of a positional relationship between a first camera device, a second camera device, and an automation device. FIG. 3 is a flowchart of a transaction behavior detection method according to an embodiment of the present invention.

Claims (10)

一種交易行為偵測系統,用於偵測操作一自動化設備之一使用者,所述的系統包括:一第一攝像裝置,用以取得一偵測區域的一第一影像,其中該偵測區域中裝設有該自動化設備,且該第一影像中包含該使用者之全身;一第二攝像裝置,用以取得操作該自動化設備之該使用者的一第二影像,且該第二影像中包含該使用者之臉部正面;一影像分析模組,電性連接至該第一攝像裝置及該第二攝像裝置,該影像分析模組用以接收該第一影像及該第二影像,分析該第一影像以產生一體態評估,且分析該第二影像以產生一情緒評估;一判斷模組,電性連接至該影像分析模組,該判斷模組用以根據該體態評估及該情緒評估綜合判斷以產生一互動策略;以及一訊息遞送模組,電性連接該判斷模組,該訊息遞送模組用以根據該互動策略選擇性地發出一提示訊息或一警示訊息。A transaction behavior detection system for detecting a user operating an automated device, the system includes: a first camera device for obtaining a first image of a detection area, wherein the detection area The automatic equipment is installed therein, and the first image includes the entire body of the user; a second camera device is used to obtain a second image of the user who operates the automatic equipment, and the second image is Including the front face of the user's face; an image analysis module electrically connected to the first camera device and the second camera device, the image analysis module is used to receive the first image and the second image and analyze The first image is used to generate an integrated assessment, and the second image is analyzed to generate an emotional assessment; a judgment module is electrically connected to the image analysis module, and the judgment module is used to evaluate the emotion and the mood according to the posture assessment. Evaluate the comprehensive judgment to generate an interactive strategy; and a message delivery module electrically connected to the judgment module, the message delivery module is used to selectively issue a prompt message or a message according to the interactive strategy Shows a message. 如請求項1所述的交易行為偵測系統,更包括一記錄模組電性連接至該影像分析模組及該判斷模組,該記錄模組用以收集該影像分析模組針對該使用者所產生之該體態評估及該情緒評估;且該判斷模組更用以根據該記錄模組中關聯於該使用者之一歷史體態評估記錄及一歷史情緒評估記錄產生該互動策略。The transaction behavior detection system according to claim 1, further comprising a recording module electrically connected to the image analysis module and the judgment module. The recording module is used to collect the image analysis module for the user. The posture evaluation and the emotion evaluation are generated; and the judgment module is further configured to generate the interaction strategy according to a historical posture evaluation record and a historical emotion evaluation record associated with the user in the recording module. 如請求項1所述的交易行為偵測系統,該體態評估包括該使用者之一體型及該使用者之一走路姿態。According to the transaction behavior detection system described in claim 1, the posture assessment includes a figure of the user and a walking posture of the user. 如請求項1所述的交易行為偵測系統,該情緒評估包括一情緒類型,該情緒類型屬於開心、不屑、憤怒、恐懼、驚訝、憎惡及悲傷其中之至少一。According to the transaction behavior detection system of claim 1, the emotion evaluation includes an emotion type, and the emotion type belongs to at least one of happy, disdain, anger, fear, surprise, hatred, and sadness. 如請求項1所述的交易行為偵測系統,該影像分析模組更包括分析該第二影像在一第一時間點及一第二時間點之一臉部特徵點變化量,且該情緒評估更包括根據該臉部特徵點變化量計算的一表情變化指數。According to the transaction behavior detection system described in claim 1, the image analysis module further includes analyzing a facial feature point change amount of the second image at a first time point and a second time point, and the emotion evaluation It further includes an expression change index calculated according to the facial feature point change amount. 如請求項1所述的交易行為偵測系統,其中該提示訊息係一螢幕顯示之文字訊息,該警示訊息係一手機簡訊。The transaction behavior detection system according to claim 1, wherein the prompt message is a text message displayed on the screen, and the warning message is a mobile phone message. 如請求項1所述的交易行為偵測系統,更包括一第一感測器電性連接至該第一攝像裝置,該第一感測器用以偵測該使用者是否已進入該偵測區域,並且當測得該使用者進入該偵測區域時,啟動該第一攝像裝置開始取得該第一影像。The transaction behavior detection system according to claim 1, further comprising a first sensor electrically connected to the first camera device, the first sensor is used to detect whether the user has entered the detection area And when it is detected that the user enters the detection area, the first camera device is activated to start acquiring the first image. 如請求項1所述的交易行為偵測系統,更包括一第二感測器電性連接至該第二攝像裝置,該第二感測器用以偵測該使用者是否已操作該自動化設備,並且當測得該自動化設備被操作時,啟動該第二攝像裝置開始取得該第二影像。The transaction behavior detection system according to claim 1, further comprising a second sensor electrically connected to the second camera device, the second sensor is used to detect whether the user has operated the automated device, And when it is measured that the automated equipment is operated, the second camera device is started to start acquiring the second image. 一種交易行為偵測方法,用於偵測操作一自動化設備之一使用者,包括:取得一偵測區域的一第一影像,其中該偵測區域中裝設有該自動化設備,且該第一影像中包含該使用者之全身;取得操作該自動化設備之該使用者的一第二影像,且該第二影像中包含該使用者之臉部正面;分析該第一影像以產生一體態評估;分析該第二影像以產生一情緒評估;根據該體態評估及該情緒評估綜合判斷以產生一互動策略;以及根據該互動策略選擇性地發出一提示訊息或一警示訊息。A transaction behavior detection method for detecting a user operating an automated device includes: acquiring a first image of a detection area, wherein the detection area is provided with the automatic device, and the first The image includes the entire body of the user; obtaining a second image of the user operating the automated equipment, and the second image includes the front of the user's face; analyzing the first image to generate an integrated assessment; Analyze the second image to generate an emotional assessment; generate an interactive strategy based on the comprehensive assessment of the posture assessment and the emotional assessment; and optionally issue a prompt message or a warning message according to the interactive strategy. 如請求項9所述的交易行為偵測方法,更包括:在產生該情緒評估之前,分析該第二影像在一第一時間點及一第二時間點之一臉部特徵點變化量,且該情緒評估更包括根據該臉部特徵點變化量計算的一表情變化指數。The method for detecting transaction behavior according to claim 9, further comprising: analyzing a change amount of a facial feature point of the second image at a first time point and a second time point before generating the sentiment assessment, and The emotion evaluation further includes an expression change index calculated according to a change amount of the facial feature point.
TW107111707A 2018-04-02 2018-04-02 System and method for detecting trading behavior TWI671701B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
TW107111707A TWI671701B (en) 2018-04-02 2018-04-02 System and method for detecting trading behavior

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
TW107111707A TWI671701B (en) 2018-04-02 2018-04-02 System and method for detecting trading behavior

Publications (2)

Publication Number Publication Date
TWI671701B TWI671701B (en) 2019-09-11
TW201942849A true TW201942849A (en) 2019-11-01

Family

ID=68618947

Family Applications (1)

Application Number Title Priority Date Filing Date
TW107111707A TWI671701B (en) 2018-04-02 2018-04-02 System and method for detecting trading behavior

Country Status (1)

Country Link
TW (1) TWI671701B (en)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
TWI730473B (en) * 2019-10-25 2021-06-11 中國信託商業銀行股份有限公司 Financial service system and warning method

Family Cites Families (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6731778B1 (en) * 1999-03-31 2004-05-04 Oki Electric Industry Co, Ltd. Photographing apparatus and monitoring system using same
TW200713137A (en) * 2005-09-28 2007-04-01 chong-de Liao Multi-functional monitoring system
CN101266704B (en) * 2008-04-24 2010-11-10 张宏志 ATM secure authentication and pre-alarming method based on face recognition
JP5877760B2 (en) * 2012-06-01 2016-03-08 日立オムロンターミナルソリューションズ株式会社 Crime prevention system and crime prevention method
CN105913559B (en) * 2016-04-06 2019-03-05 南京华捷艾米软件科技有限公司 A kind of ATM in bank intelligent control method based on body-sensing technology
CN206039795U (en) * 2016-09-14 2017-03-22 浙江维融电子科技股份有限公司 Self service bank's safe alarm system

Also Published As

Publication number Publication date
TWI671701B (en) 2019-09-11

Similar Documents

Publication Publication Date Title
US11354991B2 (en) Security system, security method, and non-transitory computer readable medium
RU2753959C2 (en) Video analysis module and method for generating materials for investigating user's behavior
US10460300B2 (en) Method of preventing fraud and theft during automated teller machine transactions and related system
AU2017201198A1 (en) Video Analytics System
CN103514438A (en) System and method for identifying human face
CN101556717A (en) ATM intelligent security system and monitoring method
JP6915542B2 (en) Information processing equipment, notification system, information transmission method and program
JP2018093283A (en) Monitoring information gathering system
CN103714631A (en) ATM intelligent monitoring system based on human face recognition
KR101957677B1 (en) System for learning based real time guidance through face recognition and the method thereof
US20160125404A1 (en) Face recognition business model and method for identifying perpetrators of atm fraud
JP2014191416A (en) Service user confirmation apparatus
Sujith Crime detection and avoidance in ATM: a new framework
JP2960490B2 (en) Automatic cash transaction system and automatic cash transaction equipment
TWI671701B (en) System and method for detecting trading behavior
JPH0535992A (en) Alarming device
JP2010238204A (en) Monitoring device and monitoring method
JP6573185B1 (en) Information processing system, information processing method, and program
JP2005242869A (en) Automatic teller system and automatic teller machine
CN111144181A (en) Risk detection method, device and system based on background collaboration
JP6206627B1 (en) Information processing apparatus, control method, and program
CN107918990A (en) A kind of antitheft system for supermarket and method
TWM565362U (en) System for detecting trading behavior
WO2020063020A1 (en) Method, device and system for identifying object
JP2020187518A (en) Information output device, method, and program