JPH09161102A - Action recognizing type automatic ticket issuing device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an action recognizing type automatic ticket issuing device with which induced guidance corresponding to the action pattern of user can be executed by recognizing the action of user while using image processing. SOLUTION: When an approach detector 2 detects the approach of user, an ITV camera 3 shoots the user and outputs the image to an action recognizing processing part 8. The action recognizing processing part 8 segments the area, where the prescribed section of user is shot, out of the input image and generates a control output by recognizing/classifying the action pattern of user based on the time sequential image data of that area. According to the control output, a guidance processing part 9 outputs the prescribed induced guidance corresponding to the recognized action pattern from a touch panel 4 and a speaker 7.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an action recognition type automatic ticket vending apparatus which recognizes a user's action using image processing and guides an operation procedure in accordance with the action pattern.

[0002]

2. Description of the Related Art As a conventional automatic ticket vending machine, for example,
It is widely used in automatic ticket vending machines, etc. that are installed at stations and issue tickets. Further, in recent years, various functions have been added to such an automatic ticket vending machine, and the automatic ticket vending machine is becoming multi-functional. As a result, the operation procedure is often complicated, and it may be difficult for a person who is unfamiliar with the operation of the device to smoothly perform the operation. Therefore, an automatic ticket vending apparatus has been proposed that guides the user's operation by, for example, performing operation guidance by a display or voice so that even an unfamiliar person can easily operate the equipment.

[0003]

However, in such an automatic ticket vending apparatus for guiding and guiding the conventional operation procedure, uniform guiding guidance is provided regardless of the user's will and psychological state. Is. That is, the state of the user (will, psychology, etc.), for example, whether the user is thinking without knowing the operation method, is trying to perform the wrong operation, or
I know the operation method, but the operation is guided regardless of whether I am looking for money. Therefore, the operation guidance that the user really needs at that time is not always provided, and conversely, there is a risk that the guidance guidance is provided to impose a psychological burden on the user.

Therefore, as one means for realizing an apparatus having an effective human interface that does not impose a psychological burden on the user, it is necessary to understand the state of the user on the apparatus side and perform appropriate guidance. It is possible to To do this, the user's condition (will, psychology, etc.)
Is considered to appear in body movements, that is, in behavior, so it is necessary to enable the automatic ticket vending machine to measure and recognize the behavior of the user.

In the automatic transaction machine known in Japanese Patent Laid-Open No. 7-49969, the presence position of the hand of the operator (user) is detected by a plurality of sensors installed at predetermined positions on the customer service surface, and this detection is performed. There has been proposed an apparatus configured to guide an operator to an appropriate operation based on the result. According to this automatic transaction machine, for example, when the operator tries to perform a predetermined operation by reaching out his hand, the sensor detects that the operator's hand is located in a place where it should not be. By guiding and guiding the appropriate operating position according to the detected position of the hand, even people who are unfamiliar with the operation of automatic transaction machines and automatic transaction machines with many operation steps can easily operate without confusion. It is the one. However, this automated teller machine only detects whether or not the user's hand is at a predetermined position, and it is hard to say that the measurement and recognition of the user's behavior are carried out. It is not sufficient to provide accurate guidance and guidance that corresponds to the will and psychological state of.

The present invention has been made paying attention to the above points, and automatically recognizes and processes the behavior of the user by using image processing, and is effective in response to the behavior pattern of the user. It is an object of the present invention to provide an action recognition type automatic ticket vending device that provides guidance.

[0007]

Therefore, according to the present invention, there is provided an automatic ticket vending apparatus for guiding operation procedures, the approach detecting means outputting an approach detection signal when a user approaches the automatic ticket vending apparatus. An image input unit that captures the user by using the approach detection signal as a start trigger, and a measurement region cutout that cuts out a region in which a predetermined region to be measured by the user is captured from an input image captured by the image input unit Means, an action pattern recognition means for recognizing the user's action pattern from the time-series image data of the cut-out region and classifying the action pattern into a preset action pattern, and the user's action pattern classified by the action pattern recognition means And a guidance output means for outputting a predetermined guidance.

According to this structure, when the user approaches the apparatus and is detected by the approach detection means, the approach detection signal is output to the image input means, and the image input means uses the approach detection signal as a start trigger. Is photographed, and the photographed image is output to the measurement site cutout means. The measurement portion cutout unit specifies and cuts out an image region in which a predetermined portion of the user has been shot, as a measurement target for recognizing the user's action pattern, from the input image shot by the image input unit. This measurement site is cut out for the input image that is continuously sent, and the action pattern recognition means recognizes the action pattern of the user based on the time-series image data indicating the continuous change of the cut out region. The recognized behavior patterns are classified into preset behavior patterns. When the user's action pattern is recognized and classified, predetermined guidance information corresponding to the action pattern is transmitted from the guidance output means to the user. The user operates according to the guidance.

[0009]

DESCRIPTION OF THE PREFERRED EMBODIMENTS One embodiment of the present invention will be described below with reference to the drawings. In the present embodiment, for example, a case where the action recognition type automatic ticket vending apparatus of the present invention is applied to an automatic ticket vending machine that performs ticket issuing processing will be described. In FIG.
The functional block diagram of the automatic ticket vending machine 1 which concerns on this embodiment is shown, and the external view of the automatic ticket vending machine 1 is shown in FIG.

In FIG. 2, the automatic ticket vending machine 1 has a customer service surface A.
In addition, an approach detector 2 as an approach detection unit that outputs an approach detection signal when the user approaches the automatic ticket vending machine 1 and an image input unit that starts photographing the user when the approach detection signal is generated, for example, , Industrial television cameras 3
(Hereinafter referred to as ITV camera 3), a section of a ticket, a fare, an operation procedure, and the like, and a touch panel 4 on which a user touches the display screen to select a boarding section, an operation method, and the like. And a charge slot 5 into which the purchase price of the ticket is inserted, and an outlet 6 from which the ticket and change are discharged.
And a speaker 7 for outputting voice guidance.

In the inside of the automatic ticket vending machine 1, as shown in FIG. 1, an action recognition processing section 8 for recognizing a user's action pattern based on an input image taken by the ITV camera 3.
And a guidance processing unit 9 that controls the display on the touch panel 4 and the speaker 7 and the output of voice guidance guidance, a ticketing processing unit 10 that issues a ticket, and a change amount that detects the amount of the fare inserted by the user. A charge processing unit 11 that discharges a ticket, a CPU 12 that controls the operation of each unit of the automatic ticket vending machine 1, a ROM 13 that stores a control program, a RAM 14 that stores and reads data relating to the control of the automatic ticket vending machine 1, and each unit. An interface circuit 15 for connection is provided.

The CPU 12, via the interface circuit 15, the action recognition processing section 8, the guidance processing section 9, the ticket issuing processing section 1
0, charge processing unit 11, ROM13 and RAM14 are connected,
Each part is driven and controlled according to a control program stored in the ROM 14. The action recognition processing unit 8 is connected to the ITV camera 3 and cuts out a region where a measurement target part of a user, which will be described later, is photographed from an input image photographed by the ITV camera 3,
The behavior pattern of the user is recognized based on the time-series data of the cut out area.

The guidance processing unit 9 is connected to the touch panel 4 and the speaker 7, and corresponds to the user's action pattern recognized by the action recognition processing unit 8 according to a control table described later stored in the RAM 14 in advance. The guidance is displayed on the touch panel 4 or voice output to the speaker 7.
The ticket issuing processor 10 issues a ticket for the boarding section selected by the user on the touch panel 4 and releases the ticket to the outlet 6.

The charge processing unit 11 detects the amount of money inserted into the charge insertion port 5, and releases the change corresponding to the difference between the charge of the boarding section selected on the touch panel 4 and the input amount into the emission port 6. Therefore, the action recognition processing unit 8 functions as a measurement site cutout unit and an action pattern recognition unit, and the guidance processing unit 9, the touch panel 4, and the speaker 7 function as guidance output unit.

Next, the operation of this embodiment will be described. FIG. 3 is a flowchart showing the operation from the time when the user approaches the automatic ticket vending machine 1 until the ticket is issued. In FIG. 3, when the user approaches the customer service surface A of the automatic ticket vending machine 1, in step 1 (indicated by S1 in the figure, the same applies hereinafter), the approach detector 2 detects the approach of the user and outputs an approach detection signal. Is output to the ITV camera 3.

In step 2, the ITV camera 3 starts the photographing of the user located in front of the customer service surface A by using the input of the approach detection signal as a start trigger, and outputs the photographed image signal to the action recognition processing section 8. In step 3, the action recognition processing unit 8 recognizes the action pattern of the user from the time-series input images continuously shot by the ITV camera 3, and generates a control output corresponding to the recognized action pattern.

Here, the processing operation of the action recognition processing section 8 in step 3 will be described in detail with reference to the functional block diagram shown in FIG. In FIG. 4, when an image is input to the action recognition processing unit 8 from the ITV camera 3, the action recognition processing unit 8 uses, as a parameter for recognizing the action pattern of the user, the part of the user to be measured, for example, the part. ,
Image regions in which the eyes, face, arms, etc. are photographed are specified from the input image region, and the image region of the measurement site is cut out. As an example, the case where the user's eyes are the measurement target will be specifically described. Regarding the method of detecting eye movements,
J. Daimon et al .: Automatic detection of driver's line of sight by moving image processing, Ergonomics, Vol.31, No.1, 39 to 50, 1995. etc., so the outline of the detection method is explained below.

First, an area in which the face part of the user is photographed is specified from the input image area based on the brightness difference of each pixel. Then, a template including both eyes and eyebrows is matched to determine the positions of both eyes from the face area, and an area including both eyes and eyebrows is cut out. Binarization processing is performed with a threshold value such that both eyes and eyebrows become black pixels in the clipped area, the connected portion of black pixels is labeled, and the area other than both eyes is removed to extract the eye area. Further, the edge of the extracted eye region is detected, and the center position of the black eye (cornea) is detected in each of the left and right eye regions based on the edge image. In this manner, the detection of the center positions of both eyes is repeatedly performed for each input image continuously sent from the ITV camera 3 to acquire time-series image data indicating a continuous change of the center positions of both eyes. .

In this way, when the time-series image data of the eyes, face, arms, etc. is acquired, the change of each measurement site is judged based on each time-series image data. For example, in the case of determining the user's eye movement based on the time-series image data indicating the continuous change of the center positions of both eyes, the center positions of both eyes of the input images that are temporally before and after are compared to each other. The amount of movement of the user is calculated, the direction of the line of sight is determined from the amount of change to estimate where the user is looking, and the eye movement is determined according to the direction in which the line of sight has moved. Similarly, for the face and the arm, the position and movement of the face and the arm are determined using the respective time-series image data. The behavior pattern of the user is recognized based on the change of each measurement site determined in this way, and is classified into a preset behavior pattern. In this classification, for example, the behavior of the user of the automatic ticket vending machine 1 is investigated in advance, and a plurality of behavior patterns that can occur at each operation stage (fee entry stage, section selection stage, etc.) are set and stored in the RAM 14. I'll do it. When the user's action pattern is actually recognized, the recognized action pattern is compared with the action pattern stored in the RAM 14 and classified into a matching or similar action pattern. For example, in the charge input stage, the user's line of sight is not located in the direction of the charge input port 5 and moves in the operation surface A of the touch panel 4 or the discharge port 6 and the action pattern is such that the arm is stationary. If it is recognized, the user does not know the operation method or is classified into the action pattern when looking for the charge slot 5. When the user's arm extends in the direction of the touch panel 4 and an action pattern in which the user's arm is moving near the display screen is recognized, the action pattern is classified when the user makes a mistake in the operation procedure. . Alternatively, when the user's line of sight is located away from the operation surface A in the direction of the user's body and an action pattern in which the arm is moving toward the body without approaching the automatic ticket vending machine 1 is recognized. , Users are categorized into behavior patterns when preparing or looking for money.

When the user's action pattern is recognized and classified, the action recognition processing section 8 refers to the control table stored in the RAM 14 in advance and outputs a control output in order to provide guidance according to the action pattern. Occur. In the control table, the control content for driving the guidance processing unit 9 and outputting predetermined guidance is set for each action pattern of each operation stage. The control output corresponding to the action pattern recognized according to this control table is output from the action recognition processing unit 8 to the guidance processing unit 9 via the interface circuit 15. Specifically, in the example of the above-mentioned action pattern, the control of the control table corresponding to the action pattern when the operation method is unknown or when the charge slot 5 is searched and the action pattern when the operation procedure is wrong The content is a control that causes the guidance processing unit 9 to output guidance guidance for prompting the charging of a fee and guidance guidance for indicating the position of the charging slot 5. Also, for the behavior pattern when preparing or looking for money,
Control is performed to put the guidance processing unit 9 in a standby state so that the next guidance is not performed until money is prepared and a fee is to be paid.

In this way, the control output is generated from the action recognition processing section 8, the operation of step 3 is completed, and the process proceeds to step 4. In step 4, the guidance processing unit 9 receives the control output from the action recognition processing unit 8 and converts the control content into display guidance on the touch panel 4 or voice guidance to the speaker 7 for proper guidance. Communicate guidance to users.

In step 5, the user performs an operation such as charging a fee according to the guidance given in step 4. Until the ticket issuing process is completed in this way,
The operation of step 5, that is, the acquisition of the input image, the recognition of the user's action pattern, the output of the guidance information, and the operation of the user are repeatedly executed. The ticket processing unit 10 and the fee processing unit
The operation of 11 is the same as the operation performed in the conventional automatic ticket vending machine or the like, and thus the description thereof is omitted here.

As described above, according to this embodiment, the automatic ticket vending machine 1 photographs and processes the behavior of the user, and automatically recognizes and classifies the behavior pattern of the user based on the processed time-series data. By doing so, it is possible to implement effective guidance and guidance that corresponds to the user's will and psychological state. Therefore, it is possible to prevent the guidance guidance that imposes a psychological burden on the user, and automatically perform the guidance guidance that the user most needs.

In the above embodiment, the measurement target part for recognizing the user's behavior pattern is the eyes, face, arms, etc., but the measurement target part can be set arbitrarily as necessary. . Further, although the approach detector is configured to use the approach detector, the approach detector is not limited to this. For example, instead of using the approach detector, an input image is captured I
A configuration in which the TV camera detects the approach of the user is also applicable. In addition, the guidance output means is not limited to the touch panel and the speaker.

[0025]

As described above, according to the present invention, the action of the user is photographed by the image inputting means, and the action pattern of the user is automatically recognized by the measuring portion cutting means and the action pattern recognizing means. By performing guidance guidance corresponding to the action pattern, it is possible to automatically perform appropriate guidance guidance according to the state of the user. Therefore, even for a user who is not accustomed to the operation, or for a device having a complicated operation procedure, effective guidance guidance can be performed without imposing a psychological burden on the user.

[Brief description of the drawings]

FIG. 1 is a functional block diagram of an automatic ticket vending machine according to an embodiment of the present invention.

FIG. 2 is an external view of an automatic ticket vending machine according to the above embodiment.

FIG. 3 is a flowchart for explaining the operation of the embodiment;

FIG. 4 is a functional block diagram illustrating an operation of an action recognition processing unit according to the above embodiment.

[Explanation of symbols]

 1 Automatic ticket vending machine 2 Approach detector 3 Industrial television camera (ITV camera) 4 Touch panel 7 Speaker 8 Action recognition processing section 9 Guidance processing section

 ─────────────────────────────────────────────────── ─── Continuation of the front page (72) Naoya Kobayashi 1-13-8 Kamikizaki, Urawa-shi, Saitama Nihon Signal Co., Ltd. Yono Works (72) Inventor Hiroyuki Wada 1-1-13 Kamikizaki, Urawa-shi, Saitama No. 8 Japan Signal Co., Ltd. Yono Works (72) Inventor Makoto Yagi 1-13-8 Kamikizaki Urawa City, Saitama Prefecture Japan Signal Co., Ltd. Yono Works

Claims (1)

[Claims] 1. An automatic ticket vending apparatus for guiding operation procedures, the approach detecting means outputting an approach detection signal when a user approaches the automatic ticket vending apparatus, and the utilization using the approach detection signal as a start trigger. Image capturing means for capturing an image of a person, measurement region clipping means for clipping a region in which a predetermined region to be measured by the user is photographed from an input image photographed by the image inputting device, and a time series of the clipped region An action pattern recognition unit that recognizes the user's action pattern from image data and classifies it into a preset action pattern, and outputs a predetermined guidance corresponding to the user action pattern classified by the action pattern recognition unit An action recognition type automatic ticket vending machine, characterized by comprising: