WO2023243172A1 - 監視装置及び監視装置方法 - Google Patents

監視装置及び監視装置方法 Download PDF

Info

Publication number
WO2023243172A1
WO2023243172A1 PCT/JP2023/011010 JP2023011010W WO2023243172A1 WO 2023243172 A1 WO2023243172 A1 WO 2023243172A1 JP 2023011010 W JP2023011010 W JP 2023011010W WO 2023243172 A1 WO2023243172 A1 WO 2023243172A1
Authority
WO
WIPO (PCT)
Prior art keywords
command
unit
operator
person
monitoring device
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/JP2023/011010
Other languages
English (en)
French (fr)
Japanese (ja)
Inventor
武史 金川
優宜 松井
敏之 枝川
啓允 今井
英明 梅村
啓生 堀内
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Hitachi Channel Solutions Corp
Original Assignee
Hitachi Channel Solutions Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Hitachi Channel Solutions Corp filed Critical Hitachi Channel Solutions Corp
Priority to JP2024528306A priority Critical patent/JP7849476B2/ja
Priority to US18/874,992 priority patent/US20250371880A1/en
Publication of WO2023243172A1 publication Critical patent/WO2023243172A1/ja
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/20—Movements or behaviour, e.g. gesture recognition
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/20—Analysis of motion
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/20—Analysis of motion
    • G06T7/246—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/70—Determining position or orientation of objects or cameras
    • G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/50—Context or environment of the image
    • G06V20/52—Surveillance or monitoring of activities, e.g. for recognising suspicious objects
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16—Human faces, e.g. facial parts, sketches or expressions
    • G06V40/174—Facial expression recognition
    • G—PHYSICS
    • G07—CHECKING-DEVICES
    • G07D—HANDLING OF COINS OR VALUABLE PAPERS, e.g. TESTING, SORTING BY DENOMINATIONS, COUNTING, DISPENSING, CHANGING OR DEPOSITING
    • G07D11/00—Devices accepting coins; Devices accepting, dispensing, sorting or counting valuable papers
    • G—PHYSICS
    • G07—CHECKING-DEVICES
    • G07D—HANDLING OF COINS OR VALUABLE PAPERS, e.g. TESTING, SORTING BY DENOMINATIONS, COUNTING, DISPENSING, CHANGING OR DEPOSITING
    • G07D11/00—Devices accepting coins; Devices accepting, dispensing, sorting or counting valuable papers
    • G07D11/40—Device architecture, e.g. modular construction
    • G—PHYSICS
    • G07—CHECKING-DEVICES
    • G07F—COIN-FREED OR LIKE APPARATUS
    • G07F19/00—Complete banking systems; Coded card-freed arrangements adapted for dispensing or receiving monies or the like and posting such transactions to existing accounts, e.g. automatic teller machines
    • G07F19/20—Automatic teller machines [ATMs]
    • G07F19/207—Surveillance aspects at ATMs
    • G—PHYSICS
    • G08—SIGNALLING
    • G08B—SIGNALLING SYSTEMS, e.g. PERSONAL CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS
    • G08B15/00—Identifying, scaring or incapacitating burglars, thieves or intruders, e.g. by explosives
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N7/00—Television systems
    • H04N7/18—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30196—Human being; Person
    • G06T2207/30201—Face
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30232—Surveillance

Definitions

  • the present invention relates to a monitoring device and a monitoring method for monitoring an operator of a transaction device that handles money and the surroundings thereof.
  • Patent Document 1 With the technology typified by Patent Document 1, it is not possible to identify and respond to a wide variety of frauds or to provide support according to the operator's condition. For these reasons, the reality is that monitoring of transaction devices has been limited to simply taking and recording images and using them for subsequent investigations.
  • one of the typical monitoring devices of the present invention includes an imaging unit whose imaging range includes an operator of a transaction device that handles money and the surroundings of the operator, and an imaging unit that captures an image of the imaging unit.
  • a posture detection unit that detects joint positions relative to the skeleton of an image of a person included in the results; a determination unit that determines the motion of the person based on the detection result of the posture detection unit; and a determination unit that determines the motion of the person based on the determination result of the determination unit.
  • a command unit that generates a command indicating a response to the person, and the determination unit includes a detection result of the posture detection unit and operator movement pattern data predefined regarding the operator's movement.
  • one of the representative monitoring methods of the present invention includes an imaging step in which the monitoring device takes an image of an operator of a transaction device handling money and the surroundings of the operator, and an imaging result included in the imaging step.
  • a posture detection step of detecting joint positions with respect to the skeleton of an image of the person; a determination step of determining the motion of the person based on the detection result of the posture detection step; and a determination step of determining the motion of the person based on the determination result of the determination step.
  • a command step for generating a command indicating a response to the posture detection step, and the determination step is performed by comparing the detection result of the posture detection step with predefined operator motion pattern data regarding the motion of the operator.
  • the motion of the person is determined, and the command step generates the command with reference to command pattern data in which the motion and the command are associated with each other.
  • Explanatory diagram for monitoring automatic teller machines Configuration diagram showing the configuration of an ATM Flowchart showing ATM processing procedure
  • Explanatory diagram for details of command pattern data Specific examples of images and actions (Part 1) Specific examples of images and actions (Part 2) Specific examples of images and actions (part 3) Specific examples of images and actions (part 4) Specific examples of images and actions (Part 5) Configuration diagram showing the configuration of the ATM of Example 2 Specific example of operation support (Part 1) Specific example of operation support (Part 2) Specific example of operation support (part 3) Diagram for sharing fraud history Illustration of object detection Illustration of advertisement display
  • the ATM 10 includes an imaging unit whose imaging range includes the operator and the surrounding area.
  • the ATM 10 detects the skeleton of the operator and the skeletons of surrounding people from an image that is a result of imaging by the imaging unit.
  • the skeleton of the operator indicates the posture of the operator.
  • the skeletons of surrounding persons indicate the postures of surrounding persons.
  • the ATM 10 determines the actions of the operator and surrounding people based on the operator's skeleton, the skeletons of surrounding people, the transaction status, etc., and generates commands indicating how to respond to the operator and surrounding people based on the judgment results.
  • Directives include those related to reporting when an operator is committing fraud, alerting the operator when nearby persons are acting suspiciously, and providing operational guidance to the operator.
  • FIG. 2 is a configuration diagram showing the configuration of the ATM 10.
  • the ATM 10 includes a monitoring control section 21, a storage section 22, an imaging section 23, a vibration sensor 24, a microphone 25, a transaction control section 31, a display operation section 32, a speaker 33, a money storage section 34, a deposit/withdrawal section 35, and the like.
  • the transaction control unit 31 is a control unit that processes various transactions including transactions involving money. Transactions include deposits, withdrawals, transfers, balance inquiries, passbook entries, etc. Note that the transaction control section 31 can be connected to a card reader, a bankbook processing section, a communication section, a receipt printer, etc. (not shown), and can use these as necessary in the transaction.
  • the display operation section 32 is connected to the transaction control section 31 and performs display output to the operator and reception of input from the operator.
  • the display operation section 32 includes a touch panel display, buttons, and the like.
  • the speaker 33 is connected to the transaction control unit 31 and outputs audio to the operator.
  • the monitoring control section 21 is connected to a transaction control section 31 , a storage section 22 , an imaging section 23 , a vibration sensor 24 , and a microphone 25 .
  • the imaging unit 23 is a camera that includes the operator and the surrounding area of the operator in its imaging range, and outputs an image that is the imaging result to the monitoring control unit 21 .
  • the vibration sensor 24 detects vibrations of the ATM 10 and outputs the detection results to the monitoring control unit 21.
  • the microphone 25 collects sounds around the ATM 10 and outputs the sound collection results to the monitoring control unit 21 .
  • the storage unit 22 is, for example, a hard disk device, and stores operator movement pattern data 22a, peripheral person movement pattern data 22b, transaction operation procedure data 22c, and command pattern data 22d.
  • the transaction operation procedure data 22c is data indicating the transaction operation procedure of the ATM 10.
  • the operating procedure indicates the order of operations when performing the transaction.
  • the operational steps for withdrawal include "accepting a card,” “inputting an amount,” and "removing money.” Whether each operation has been executed can be acquired from the transaction control unit 31.
  • the motion for each operation can also be registered in the operator motion pattern data 22a.
  • the command pattern data 22d is data that associates the actions of the operator or surrounding persons with commands. Details of the command pattern data 22d will be described later.
  • the monitoring control unit 21 is a control unit that controls monitoring of the ATM 10, and may be realized by, for example, a CPU (Central Processing Unit).
  • the monitoring control unit 21 realizes the functions of the attitude detection unit 21a, the determination unit 21b, and the command unit 21c.
  • the posture detection unit 21a detects joint positions regarding the skeleton of the person's image included in the imaging result of the imaging unit 23. Specifically, the posture detection unit 21a extracts a person's image from the image and detects the joint positions of the skeleton. At this time, the distance to the person may be identified based on the position of the person's image. For example, the operator can be identified as being near the ATM 10 based on the size of the image. Regarding surrounding people, the distance to the ATM 10 can be estimated from the size of the image in the image, the position of the feet, etc.
  • the determination unit 21b determines the motion of the person based on the detection result of the posture detection unit 21a.
  • the determining unit 21b determines the operator's motion by comparing the detection result of the posture detecting unit 21a and the operator's motion pattern data 22a. Further, the determination unit 21b determines the motion of the surrounding person by comparing the detection result of the posture detection unit 21a and the surrounding person movement pattern data 22b.
  • the determination unit 21b can also determine the behavior of the person further using the transaction status acquired from the transaction control unit 31. For example, it can be determined that a card insertion operation is being performed based on the posture of the operating person and the state of the card reader.
  • the determination unit 21b may identify the transaction status by comparing the detection result of the posture detection unit 21a with the transaction operation procedure data 22c, and further use the identified transaction status to determine the person's movement. can. Furthermore, it is also possible to further use the outputs of the microphone 25 and the vibration sensor 24 to determine the motion of the person. As an example, if sound or vibration is detected when the operator is squatting in front of the ATM 10, it can be determined that there is a possibility that the money storage section 34 is being destroyed.
  • the command unit 21c can generate a command for a situation specified by a combination of the operator's actions and the actions of surrounding people. For example, if an operator is squatting in front of the ATM 10 and people nearby are waiting their turn, there is a high possibility that the operator is doing something that is not fraudulent, such as picking up a lost item. On the other hand, if the operator is crouching in front of the ATM 10 and people nearby are acting cautious of their surroundings, there is a possibility that the operator and the people nearby are working together to commit fraud. .
  • the command unit 21c can generate a command for at least one of reporting, warning, restraining, and calling attention when the person's action is an illegal act. Further, the command unit 21c can generate a command to support the operator's operation when the operator's operation deviates from a proper operating procedure. Furthermore, the command unit 21c can generate a command to cancel and start the power saving mode of the ATM 10 based on the approach and departure of the operator. Specific examples of these commands will be described later.
  • FIG. 3 is a flowchart showing the processing procedure of the ATM 10.
  • the ATM 10 repeatedly executes the processes of steps S101 to S109.
  • the imaging unit 23 captures an image (step S101).
  • the posture detection unit 21a detects a skeleton from the imaging result of the imaging unit 23 (step S102), and identifies the operator and surrounding persons (step S103).
  • the posture detection unit 21a detects the postures of the operator and surrounding persons from the positional relationship of the joints (step S104).
  • the command unit 21c refers to the command pattern data 22d based on the movements of the operator and surrounding persons, and generates commands if necessary (step S107). Once the command is generated, it is necessary to output the command (step S108; Yes), so the generated command is output (step S109), and the process ends. If no command has been generated (step S108; No), the process ends.
  • FIG. 4 is an explanatory diagram of details of the command pattern data.
  • the command pattern data 22d indicates that if the operator's action is "transaction operation in progress" and the actions of surrounding persons are "peeking", a command to alert the operator is generated. ing. Specifically, a command for causing the display operation unit 32 to output a display indicating that the user is being viewed from behind is suitable.
  • the command pattern data 22d indicates that there is no command if the operator's action is "transaction operation in progress" and the surrounding person's action is "mobile phone call in progress".
  • command pattern data 22d indicates that an operation support command is generated if the operator's action is "stagnation or deviation from the operation procedure". For example, if the procedure shown in the transaction operation procedure data 22c is stalled in the middle, the display operation unit 32 is caused to output a display indicating the next procedure.
  • command pattern data 22d indicates that if the operator's action is "transfer operation while talking on the phone", a withdrawal command is generated.
  • bank transfer fraud can be prevented by displaying a message such as "Please wait for a while. An attendant will arrive.” You may also make a report. Alternatively, the operator may be directly informed that there is a suspicion of bank transfer fraud.
  • command pattern data 22d indicates that there is no command if the operator's action is a short squatting motion. This is because there is a possibility that the person is picking up something that has been lost.
  • the command pattern data 22d indicates that when the operator's action is crouching for a long time, different responses are taken depending on the accuracy of the crouching determination. Squatting down for a long time may be an abnormal act such as vandalism to the coin storage unit 34. Therefore, the command pattern data 22d associates a command to call attention (for example, outputting an alarm sound from the speaker 33) when the accuracy is low, and a command to notify a security guard when the accuracy is high.
  • a command to call attention for example, outputting an alarm sound from the speaker 33
  • the command pattern data 22d indicates that a command to cancel and start the power saving mode of the ATM 10 is generated based on the approach and departure of the operator.
  • the power saving mode is a mode in which power consumption of unnecessary functions is suppressed when no operator is present.
  • the command pattern data 22d states that a command to cancel the power saving mode is generated when the approach of the operator is detected. Additionally, when the operator finishes the transaction and leaves the transaction, and there are no nearby people waiting for their turn, a command to start the power saving mode is generated.
  • the posture detection unit 21a detects the skeletal position of each person displayed on the screen, including the head, left and right shoulders, left and right arm joints, and the like.
  • the determination unit 21b determines whether each person is a person facing the ATM (an ATM operator) or other surrounding people based on the size of a triangle connecting the head and both shoulders and its position on the screen, for example. Determine whether it is a person.
  • the determining unit 21b determines that the person on the near side is an operator facing the ATM based on the size and position on the screen, and that the person is inserting a card into the ATM 10 based on the skeletal information.
  • the person on the back side is determined to be a person in the vicinity of the ATM 10, and is determined to be looking into the screen of the ATM 10.
  • the command unit 21c instructs the transaction control unit 31 to take predefined actions corresponding to each piece of information reported by the determination unit 21b. For example, since the operator of the ATM 10 inserting a card is an act within the scope of normal ATM operation, no command is generated in that case. Furthermore, since it is an abnormal act for a person in the vicinity of the ATM 10 to look into the screen, a warning is displayed on the screen of the ATM 10. Based on predefined priorities for the definition information for these multiple people, in this case, for example, priority is given to commands regarding the actions of surrounding people, and the command unit issues a warning to the host device on the screen of the ATM 10. Command to display. For example, the priority is set in the order of notification, warning, restraint, attention call, operation support, and no command, and if there are multiple commands corresponding to the determination result, the highest command may be generated.
  • the determining unit 21b determines, for example, that the person on the near side is an ATM operator based on the size and position on the screen, and that he is touching a button on the screen of the ATM 10 based on skeletal information.
  • the person on the back side is similarly determined to be a person in the vicinity of the ATM 10, and is determined to be talking on a mobile phone.
  • the command unit 21c determines that the action is within the range of normal ATM operation and no command is given. Also, if a person in the vicinity of the ATM 10 is talking on a mobile phone, this is an action that can normally be taken, so no command is given. Based on these definition information, the command unit 21c does not output a command to the transaction control unit 31.
  • the determination unit 21b determines that the person in the image is the operator of the ATM 10, and also determines that the person is about to turn around and leave.
  • the command unit 21c obtains information regarding the progress status of the operation being performed by the operator, for example, from the transaction control unit 31. For example, if information is obtained that the operator has completed a transaction, there is no need to output a command to the transaction control unit 31. Alternatively, if information is obtained that the operator has not yet taken out the withdrawn banknotes, the operator instructs the speaker 33 to issue a ring tone or the like to alert the operator.
  • the determination unit determines that the person in the image is the person facing the ATM 10, that is, the operator for convenience, based on the size and position on the screen, and determines that the person is squatting in front of the ATM 10.
  • the command unit 21c determines that the act of the operator crouching is an abnormal act such as vandalism of a safe, and instructs the transaction control unit 31 to call a security guard, for example. For example, if the crouching time is short, the command unit 21c treats it as a normal action such as picking up something dropped and instructs the command unit 21c to do nothing, and if the time is longer than a certain level, it instructs the safe to melt.
  • the command unit 21c may issue a warning sound or prevent crime by, for example, emitting an alarm sound from the ATM, since this posture cannot necessarily be considered as a posture for vandalism. If the accuracy of the match is high, it will be considered as a posture for sabotage, and the content of the command will be changed depending on the accuracy of the match. It can be changed.
  • the determining unit 21b determines that the person in the image is an operator facing the ATM 10, and which part of the ATM 10 is being operated based on the skeleton information.
  • the area surrounded by a broken line in FIG. 9 includes a touch panel display, a switch, a card reader, a money deposit/withdrawal port, a receipt printer, and the like. If the position of the operator's hand matches any of the areas surrounded by the broken line, it can be determined which operation the operator performed. If the determined operation is different from the order shown in the transaction operation procedure data 22c, there is a possibility of an operation error, so the command unit 21c instructs the operator to perform the correct operation.
  • FIG. 10 is a configuration diagram showing the configuration of the ATM of Example 2.
  • the ATM 10 shown in FIG. 10 has a configuration in which a behavior estimation section 21d, a facial expression detection section 21e, an object detection section 21f, and a heart rate detection sensor 26 are added to the configuration shown in FIG. Further, the ATM 10 shown in FIG. 10 can communicate with the face authentication server 41 via the network.
  • the behavior estimation section 21d, the facial expression detection section 21e, and the object detection section 21f are included in the monitoring control section 21.
  • the heart rate detection sensor 26 is a sensor that detects the heart rate of the operator.
  • the behavior estimation unit 21d estimates the smoothness of the operator's operation.
  • the command unit 21c generates a command to support the operator's operation when the smoothness of the operation is insufficient.
  • the behavior estimation unit 21d estimates whether or not the operation is smooth using the stagnation of the operation, the operator's facial expression, the operator's heart rate, and the like.
  • the stagnation of an operation can be determined by acquiring the transaction status in the transaction control unit 31 and comparing the estimated time required until the next operation with the actual time required for the operation.
  • the facial expression of the operator can be detected by the facial expression detection unit 21e performing image processing on the image of the operator's face.
  • the heart rate of the operator can be detected by the heart rate detection sensor 26.
  • the behavior estimation unit 21d estimates that smoothness is low if stagnation occurs in the operation. If the operator's expression shows confusion, the behavior estimation unit 21d estimates that the smoothness is low. The behavior estimation unit 21d estimates that the smoothness is low when the heart rate of the operator is high. In addition to these, the behavior estimation unit 21d estimates the smoothness of the operation using arbitrary indicators, such as when the operator is about to perform an operation that differs from the procedure, when the operator accepts cancellation of an operation, or when a confused action is detected. can.
  • the behavior estimation unit 21d stores information regarding operations estimated to be insufficiently smooth in a predetermined storage unit. For example, by storing operations that the operator was unable to perform smoothly in the storage unit 22, this can be used to improve the interface.
  • the command unit 21c associates information indicating the external appearance characteristics of the person who committed the fraudulent act (for example, a face image or feature amount of the facial image) with information indicating the fraudulent act (for example, the type of fraud), and It is registered in the face authentication server 41, which is a device. As a result, the face authentication server 41 accumulates fraudulent history data.
  • the command unit 21c can inquire of the face authentication server 41 regarding the person imaged by the imaging unit 23. As a result of the inquiry, if there is a history of fraud, the information obtained from the new transaction is linked and registered in the face authentication server 41. For example, if a person who commits fraud uses the person's account, it is possible to link the fraudulent act to the person's identity information.
  • a command may be generated for at least one of reporting, warning, restraining, and calling attention. In other words, it is possible to report a person who has committed fraud in the past without having to make a new determination of fraud.
  • the object detection unit 21f detects an object held by a person by performing image processing on the imaging result of the imaging unit 23, and identifies the detected object.
  • the command unit 21c generates a command regarding a report and/or a warning when the object is an object related to an illegal act.
  • Objects related to fraudulent activities include items that are unnecessary for transaction operations, items that may be used to destroy the ATM 10, and the like.
  • the ATM 10 may include a display section capable of displaying advertisements.
  • the display section capable of displaying advertisements may be a part of the display operation section 32, or may be provided as a separate display section.
  • the determination unit 21b can determine the person's line of sight.
  • the command unit 21c can switch the advertisement display when the person's line of sight turns to the advertisement display display unit. Furthermore, the determination unit 21b can measure and record the time period during which the person's line of sight was directed toward the display unit for displaying advertisements.
  • 11 to 13 are specific examples of operation support.
  • the transactions illustrated in FIG. card, and receipt receipt.
  • the command unit 21c issues the guidance "Please insert your card or passbook”. ” to support the operator's operations. Additionally, if the transaction progress status is the "amount input screen" and the skeleton information indicates that the operator is unsure, the command unit 21c outputs the guidance "Please enter the amount you wish to withdraw.” By doing so, it supports the operator's operations.
  • the ATM 10 can issue appropriate guidance by comparing and determining the actions of the ATM 10 operator and at what stage of the transaction procedure the actions are taking place.
  • the guidance may be a display or sound, or may be a control that makes the part to be operated stand out, such as by making the card slot blink more intensely.
  • the ATM 10 In addition to providing operational guidance, the ATM 10 also remembers at what stage in the transaction procedure the operator was confused about the operation. Therefore, the next time the same person performs an operation, it is possible to change the guidance in the same situation to make it easier to understand the operation, and by capturing areas where many users are confused about the operation as big data, it can be used to improve the guidance. .
  • the transaction progress state is "insert card or passbook", and facial expression detection shows that the customer is confused. Therefore, the command unit 21c supports the operator's operation by outputting the guidance "Please insert your card or passbook.” The ATM 10 also remembers at what stage in the transaction procedure the operator became confused.
  • a confused or troubled face can be determined by detecting facial expressions and actions such as wrinkles between the eyebrows, drooping of the corners of the eyes, and tilting the head.
  • the transaction progress status is "insertion of card or passbook", and it is presumed that the user is having trouble due to an increase in heart rate. Therefore, the command unit 21c supports the operator's operation by outputting the guidance "Please insert your card or passbook.”
  • the ATM 10 also remembers at what stage in the transaction procedure the operator became confused.
  • Heart rate can be measured without contact by detecting minute vibrations on the body surface using millimeter wave radar.
  • the behavior estimation unit 21d detects that the heart rate has increased, the behavior estimation unit 21d infers that the person is in an anxious state, such as not knowing the procedure, or in an anxious state, such as a case of wire fraud.
  • FIG. 14 is an explanatory diagram of sharing of fraud history.
  • user A performs a fraudulent act at ATM 10a and then operates ATM 10b installed at a different location.
  • the ATM 10a registers and shares the information of the person who committed the fraudulent act with an external device, and thereafter, if any ATM detects that person, the information of that person is linked. Therefore, it is possible to accumulate information that can be used to confirm the identity of the person who committed the fraudulent act, such as account information.
  • FIG. 16 is an explanatory diagram of advertisement display.
  • the ATM 10 can display advertisements and the like. For example, a part of the display area of the display operation unit 32 used for transactions may be used to display advertisements to the operator. Further, an advertising display section may be separately provided above the ATM 10 so that it can be seen by people in the vicinity.
  • the ATM 10 detects the line of sight of the operator and surrounding people through image processing and the like. Then, when it detects that the line of sight is directed toward the advertising display area, the display content is switched and the advertisement is displayed. Furthermore, by measuring and recording the time that the line of sight is directed toward the display unit, it is possible to accumulate evaluation materials for the content of the advertisement.
  • advertisement content may be determined in conjunction with user information and transaction details. For example, conditions such as ⁇ If your account balance is low and you are making withdrawals or transfers, we will advertise a loan'' or ⁇ If your account balance is large and you are making deposits, we will guide you on asset management.'' You can use
  • the determination unit 21b determines the motion of the person by comparing the detection result of the posture detection unit 21a with operator motion pattern data 22a defined in advance regarding the motion of the operator, and determines the motion of the person.
  • 21c is characterized in that the command is generated by referring to command pattern data 22d in which the action and the command are associated with each other.
  • the determination unit 21b can determine the behavior of the person further using the transaction status acquired from the transaction device. Further, the determination unit 21b identifies the transaction status by comparing the detection result of the attitude detection unit 21a with transaction operation procedure data 22c indicating the operation procedure of the transaction device, and further utilizes the transaction status. It is possible to determine the motion of the person. By acquiring the transaction status in this manner, the operator's actions can be determined in more detail.
  • the command unit 21c generates a command regarding at least one of reporting, warning, restraining, and calling attention when the action of the person is an illegal act. Therefore, the monitoring device can appropriately deter fraudulent acts.
  • the monitoring device further includes a microphone and/or a vibration sensor, and the determination unit 21b further uses the output of the microphone and/or the vibration sensor to determine the motion of the person. Therefore, vandalism and unauthorized processing can be determined with high accuracy.

Landscapes

  • Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Multimedia (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Health & Medical Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Human Computer Interaction (AREA)
  • Finance (AREA)
  • Psychiatry (AREA)
  • Social Psychology (AREA)
  • Business, Economics & Management (AREA)
  • Accounting & Taxation (AREA)
  • Oral & Maxillofacial Surgery (AREA)
  • Signal Processing (AREA)
  • Financial Or Insurance-Related Operations Such As Payment And Settlement (AREA)
PCT/JP2023/011010 2022-06-17 2023-03-20 監視装置及び監視装置方法 Ceased WO2023243172A1 (ja)

Priority Applications (2)

Application Number Priority Date Filing Date Title
JP2024528306A JP7849476B2 (ja) 2022-06-17 2023-03-20 監視装置及び監視装置方法
US18/874,992 US20250371880A1 (en) 2022-06-17 2023-03-20 Monitoring apparatus and monitoring apparatus method

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2022-098471 2022-06-17
JP2022098471 2022-06-17

Publications (1)

Publication Number Publication Date
WO2023243172A1 true WO2023243172A1 (ja) 2023-12-21

Family

ID=89192680

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2023/011010 Ceased WO2023243172A1 (ja) 2022-06-17 2023-03-20 監視装置及び監視装置方法

Country Status (3)

Country Link
US (1) US20250371880A1 (https=)
JP (1) JP7849476B2 (https=)
WO (1) WO2023243172A1 (https=)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP7692542B1 (ja) * 2025-02-04 2025-06-13 ＰａｙＰａｙ株式会社 アプリケーションプログラム、およびサービス提供システム

Citations (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH0749915A (ja) * 1993-08-04 1995-02-21 Omron Corp 自動取引エリアの防犯装置
JPH1063919A (ja) * 1996-08-27 1998-03-06 Hitachi Ltd 音声ガイダンス付き入力方法及び装置
JP2006209163A (ja) * 2005-01-25 2006-08-10 Hitachi Omron Terminal Solutions Corp 現金自動取引装置の制御
JP2006331049A (ja) * 2005-05-26 2006-12-07 Oki Electric Ind Co Ltd 防犯監視システム
JP2007265125A (ja) * 2006-03-29 2007-10-11 Matsushita Electric Ind Co Ltd コンテンツ表示装置
JP2010257352A (ja) * 2009-04-27 2010-11-11 Oki Electric Ind Co Ltd 自動取引装置
JP2010257349A (ja) * 2009-04-27 2010-11-11 Oki Electric Ind Co Ltd 自動取引装置
JP2014016715A (ja) * 2012-07-06 2014-01-30 Hitachi Omron Terminal Solutions Corp 現金自動取引装置およびその制御方法
JP2017121286A (ja) * 2016-01-05 2017-07-13 富士通株式会社 情動推定システム、情動推定方法および情動推定プログラム
JP2018190012A (ja) * 2017-04-28 2018-11-29 大日本印刷株式会社 接客要否判定装置、接客要否判定方法、及びプログラム
WO2021186610A1 (ja) * 2020-03-18 2021-09-23 株式会社 テクノミライ デジタル・オートファイル・セキュリティシステム、方法及びプログラム

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP6710467B2 (ja) 2016-12-22 2020-06-17 株式会社日立製作所 電子投票システム

Patent Citations (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH0749915A (ja) * 1993-08-04 1995-02-21 Omron Corp 自動取引エリアの防犯装置
JPH1063919A (ja) * 1996-08-27 1998-03-06 Hitachi Ltd 音声ガイダンス付き入力方法及び装置
JP2006209163A (ja) * 2005-01-25 2006-08-10 Hitachi Omron Terminal Solutions Corp 現金自動取引装置の制御
JP2006331049A (ja) * 2005-05-26 2006-12-07 Oki Electric Ind Co Ltd 防犯監視システム
JP2007265125A (ja) * 2006-03-29 2007-10-11 Matsushita Electric Ind Co Ltd コンテンツ表示装置
JP2010257352A (ja) * 2009-04-27 2010-11-11 Oki Electric Ind Co Ltd 自動取引装置
JP2010257349A (ja) * 2009-04-27 2010-11-11 Oki Electric Ind Co Ltd 自動取引装置
JP2014016715A (ja) * 2012-07-06 2014-01-30 Hitachi Omron Terminal Solutions Corp 現金自動取引装置およびその制御方法
JP2017121286A (ja) * 2016-01-05 2017-07-13 富士通株式会社 情動推定システム、情動推定方法および情動推定プログラム
JP2018190012A (ja) * 2017-04-28 2018-11-29 大日本印刷株式会社 接客要否判定装置、接客要否判定方法、及びプログラム
WO2021186610A1 (ja) * 2020-03-18 2021-09-23 株式会社 テクノミライ デジタル・オートファイル・セキュリティシステム、方法及びプログラム

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP7692542B1 (ja) * 2025-02-04 2025-06-13 ＰａｙＰａｙ株式会社 アプリケーションプログラム、およびサービス提供システム

Also Published As

Publication number Publication date
JP7849476B2 (ja) 2026-04-21
JPWO2023243172A1 (https=) 2023-12-21
US20250371880A1 (en) 2025-12-04

Similar Documents

Publication Publication Date Title
US10629036B2 (en) Surveillance device
US9607483B2 (en) Processing automated banking transactions requiring approval
US8172134B1 (en) Automated banking machine and system that operates to cause financial transfers responsive to data read from data bearing records
US8395500B1 (en) Self-service device security alert response system
US11881088B2 (en) Hand action tracking for card slot tampering
JP5359792B2 (ja) 例外処理管理装置、例外処理管理装置の制御方法およびプログラム
US8988186B1 (en) Self-service device user asset condition alert
CN103456104B (zh) 防止犯罪系统以及防止犯罪方法
CN101510331A (zh) Atm柜员机智能型键盘输入装置
WO2023243172A1 (ja) 監視装置及び監視装置方法
JP2010072989A (ja) 電子化ジャーナルと映像の統合管理システム
JP2007280317A (ja) 不審物検知システム
JP5192324B2 (ja) 監視システム及び自動取引装置
JP2007087316A (ja) 自動取引装置及び自動取引システム
JP2010123010A (ja) 自動取引装置及びシステム
JP2007220036A (ja) 現金預払機
JP2010079754A (ja) 通報装置
US12272144B2 (en) Computer vision-assisted compliance monitoring
JP2005242869A (ja) 現金自動取引システム及び現金自動取引装置
JP2010079752A (ja) 自動取引システム
JP4349971B2 (ja) 現金自動取引装置
JP2007094588A (ja) 顔認証システム
JP7517945B2 (ja) ログインシステム、電子機器及びログイン制御方法
JP2010079753A (ja) 自動取引システム
JP2008023097A (ja) 監視装置および方法、並びにプログラム

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 23823476

Country of ref document: EP

Kind code of ref document: A1

WWE Wipo information: entry into national phase

Ref document number: 18874992

Country of ref document: US

NENP Non-entry into the national phase

Ref country code: DE

WWE Wipo information: entry into national phase

Ref document number: 2024528306

Country of ref document: JP

122 Ep: pct application non-entry in european phase

Ref document number: 23823476

Country of ref document: EP

Kind code of ref document: A1

WWP Wipo information: published in national office

Ref document number: 18874992

Country of ref document: US