JP2011237962A - Operation recognition device, information processing system, operation recognition method and operation recognition program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an operation recognition device capable of improving the recognition accuracy of operations on a body to be contacted.SOLUTION: A body to be contacted 80 is contacted by a contact object which is made to contact or applied by an operator. Contact position detection means 81 detects a position and an area where the contact object contacts a body to be contacted, according to passage of time. Recognition means 82 recognizes an event in which a change of an area position showing the location of the contact area is within the predefined length as a first event, and an event in which the change of the area position exceeds the predefined length as a second event, if a dimension of the contact area increases or decreases as time passes. Event output means 83 outputs information showing the recognized event to other devices. And the event output means 83 outputs information showing the first event and showing an event other than the second event to other devices, if the first event is recognized.

Description

  The present invention relates to an operation recognition device, an information processing system, an operation recognition method, an information processing method, and an operation recognition program, and in particular, an operation recognition device capable of recognizing a touch operation, an information processing system, an operation recognition method, an information processing method, and an operation. It relates to recognition programs.

  The touch panel that can control the device by pressing the displayed screen (hereinafter referred to as touch operation) is used by an unspecified number of people such as automated teller machines at banks because the operation is intuitive and easy to understand. Widely used mainly in systems.

  The touch panel is usually realized by combining a display device such as a liquid crystal panel and a device that detects a position touched by a user (hereinafter referred to as a position recognition device). There are various types of position recognition devices, but typical examples include a resistance film method, a capacitance method, and an optical device.

  In the resistance film method, an apparatus is used in which a conductive thin film is attached to both a glass forming a display screen and a film attached to the glass at a slight interval. In this method, a position touched by a user is detected by sensing a place where power is supplied when the finger of the user is pressed.

  In the capacitance method, a position touched by a user is detected by capturing a change in capacitance generated between the fingertip and the conductive film. Further, in the optical system, a position touched by the user is detected by photographing a light emitted from a light source and scattered by a user's finger or the like, or a portion that is blocked and does not reach the light with a camera or the like. There are also some devices that detect a contact by a plurality of fingers at the same time or detect a contact area by devising a mechanism.

  Note that the types of objects that can be detected by the position detection device generally depend on the sensor employed in the device. There are various types of objects that can be detected. For example, there are sensors that can detect a human body based on the principle, sensors that can detect a pen-like instrument called a stylus in addition to a human body, and the like.

  For example, a touch panel using a Frustrated Total Internal Reflection method (hereinafter referred to as FTIR method) is known as one method of an optical touch panel (see Patent Document 1). In the method described in Patent Document 1, light enters a waveguide such as a flat acrylic panel so as to satisfy the total reflection condition. When an object such as a finger contacts the waveguide, part of the light in the waveguide is scattered at the contact position of the object. Therefore, by detecting the phenomenon in which a part of the light is scattered using an image sensor such as a camera, for example, the touch operation and the contact position can be detected.

  Patent Document 2 describes a touch operation input device. In the input device described in Patent Document 2, when the amount of increase in the contact area per unit time is larger than the set gradient threshold, it is determined that the confirmation operation has been performed on the operation surface.

US Patent Application Publication No. 2008/179507 JP 2006-209684 A (paragraph 0007)

  On the other hand, there are many cases where the input is erroneously recognized as a touch operation due to a sensor input other than the touch operation despite the intention of detecting the touch operation. For example, in the method described in Patent Document 1, when light in the waveguide is scattered for reasons other than the touch operation, there is a possibility that it is erroneously recognized as a touch operation. For example, if any substance that scatters light is attached to the waveguide, the method described in Patent Document 1 erroneously determines that the user is in a state of constantly holding finger contact. .

  Patent Document 1 discloses a technique for preventing sensor input from being generated by an operation other than a touch operation as a method for reducing such erroneous determination. In patent document 1, the adhesion of sebum is mentioned as an example as a phenomenon which scatters light other than finger contact. In the method described in Patent Document 1, sebum adhesion is prevented by covering the waveguide with a thin resin sheet or the like.

  Specifically, a narrow air gap layer is provided between the resin sheet and the waveguide, and when the resin sheet touches the waveguide by a physical force accompanying pressing of a finger or the like, Light is scattered. In this configuration, the user does not directly touch the waveguide. Therefore, it is possible to prevent substances such as sebum from adhering to the waveguide.

  The above-described method for blocking sensor input by operations other than touch operation works effectively when sensor input other than touch operation is input information (noise input such as sebum) unnecessary for the system. However, in many cases, the user intentionally performs an operation other than the touch operation, and often wants to handle it as input information different from the touch operation. For example, in the case of a system that simultaneously detects a coating operation and a touch operation by a sensor that simultaneously detects a position where a coating material is attached in accordance with a coating operation of the coating material and a finger contact position associated with a touch operation. No method for preventing adhesion is available.

  On the other hand, Patent Document 2 discloses a method for determining that a confirmation operation (touch operation) has been performed on the operation surface based on an increase in the contact area per unit time. However, even in operations other than touch operations, there are many operations that cause an increase in contact area per unit time. For example, in the case of the application operation, as the application object is applied by the application tool, the adhesion area of the application object increases, so that the contact area detected by the sensor also increases. In other words, the operation for applying the applied product is very similar to the touch operation. Therefore, even though the application operation is performed, the operation is often mistakenly recognized as a touch operation.

  Accordingly, an object of the present invention is to provide an operation recognition device, an information processing system, an operation recognition method, an information processing method, and an operation recognition program that can improve the recognition accuracy of an operation on a contacted object.

  The operation recognizing device according to the present invention shows a contacted body that is contacted by a contact object that is contacted by an operator or a contact object that is applied by the operator, and a contact position and a contact area of the contact object with respect to the contacted object as time passes. When the contact position detecting means to detect and the area of the contact area increases or decreases with the passage of time, an event in which a change in the area position representing the position of the contact area is within a predetermined distance is referred to as a first event. Recognizing means for recognizing and recognizing an event in which the change of the region position exceeds the distance as a second event, and an event output means for outputting information representing the recognized event to another device, the event output means However, when the first event is recognized, the information indicating the first event, which is different from the information indicating the second event, is output to another device.

  According to the information processing system of the present invention, the contacted body that is contacted by the contact object that the operator makes contact with or the contact object that is applied by the operator, and the contact position and the contact area of the contact object with respect to the contacted object are displayed with time. When the contact position detecting means to detect and the area of the contact area increases or decreases with the passage of time, an event in which a change in the area position representing the position of the contact area is within a predetermined distance is referred to as a first event. Recognizing means for recognizing an event in which the change of the region position exceeds the distance as a second event, and responding means for performing a predetermined response according to the recognized event, the response means comprising: When the first event is recognized, the response is determined according to the first event, and is different from the response determined according to the second event.

  In the operation recognition method according to the present invention, the contact position of the contact object and the contact area of the contact object with respect to the contact object contacted by the contact object contacted by the operator or the contact object applied by the operator are determined. When the area of the contact area is detected with the passage of time and increases or decreases with the passage of time, an event in which the change of the area position representing the position of the contact area is within a predetermined distance is recognized as the first event. When the event where the change in the region position exceeds the distance is recognized as the second event, the information indicating the recognized event is output to the other device, and the information indicating the event is output to the other device. When one event is recognized, information indicating the first event, which is different from the information indicating the second event, is output to another device.

  According to the information processing method of the present invention, the contact position and the contact area of the contact object contacted by the contact object contacted by the contact object contacted by the operator or the contact object applied by the operator are determined. When the area of the contact area is detected with the passage of time and increases or decreases with the passage of time, an event in which the change of the area position representing the position of the contact area is within a predetermined distance is recognized as the first event. Then, an event in which the change in the region position exceeds the distance is recognized as a second event, a predetermined response is made according to the recognized event, and when the response is made, the first event is recognized In some cases, the response is determined according to the first event and is different from the response determined according to the second event.

  An operation recognition program according to the present invention is an operation recognition program applied to a computer including a contacted body that is contacted by a contact object that is touched by an operator or a contact object that is applied by the operator. Contact position detection processing for detecting the contact position and contact area of a contact object with respect to the contact body with the passage of time. When the area of the contact area increases or decreases with the passage of time, a change in the area position representing the position of the contact area occurs. Recognition processing that recognizes an event that is within a predetermined distance as a first event, recognizes an event in which the change in the region position exceeds the distance as a second event, and other information that represents the recognized event When the first event is recognized in the event output process, the information indicating the first event is the second The information different from the information indicative of the elephant, characterized thereby output to another device.

  According to the present invention, it is possible to improve the recognition accuracy of an operation on a contacted object.

It is a block diagram which shows one Embodiment of the operation recognition apparatus by this invention. It is explanatory drawing which shows the example of the state in which the light of the light source part 12 entered into the to-be-contacted body 11. It is explanatory drawing which shows the example of the reflection state of the light in the surface of the to-be-contacted body. It is explanatory drawing which shows the example of an image frame. It is a graph which shows the example of a change of a contact area and the position of an area | region when touch operation is performed. It is a flowchart which shows the example of the process which the operation recognition part 2 performs. It is explanatory drawing which shows the example of the image frame in each time. It is explanatory drawing which shows the example of a difference image. It is explanatory drawing which shows the example of a relationship between an image frame and a difference image. It is a block diagram which shows one Embodiment of the information processing system by this invention. It is a flowchart which shows the example of the process which the operation response part 1 performs. It is explanatory drawing which shows the example of operation with respect to the to-be-contacted body 11, and the display with respect to the display part. It is a block diagram which shows the example of the minimum structure of the operation recognition apparatus by this invention. It is a block diagram which shows the example of the minimum structure of the information processing system by this invention.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

Embodiment 1. FIG.
FIG. 1 is a block diagram showing an embodiment of an operation recognition apparatus according to the present invention. The operation recognition device 10 in this embodiment includes a contacted body 11, a light source unit 12, a detection unit 13, and an operation recognition unit 2. The operation recognition device 10 is connected to the operation response unit 1. The touched object 11 is touched by touching the user's finger 14 or the like, or is applied and removed by the applicator 15 with the applicator 16. Note that an object that comes into contact with the contacted object 11 such as the user's finger 14 and a substance that is applied to the contacted object 11 such as the applied object 16 will be hereinafter collectively referred to as a contacted object.

  The to-be-contacted body 11 is a transparent flat plate that can apply or wipe off the coating 16, and is realized by a resin plate such as acrylic.

  The light source unit 12 causes light outside the visible light region to enter from the side surface of the contacted body 11 so that light outside the visible light region (for example, infrared light) is guided into the contacted body 11. The light source unit 12 is realized by, for example, one or more infrared light emitting diodes (LEDs) arranged in a straight line.

  In the example shown in FIG. 1, the light source unit 12 is disposed at two places on the contacted body 11 in the upper and lower portions. However, the arrangement position of the light source unit 12 is not limited to two places above and below the contacted body 11. The light source unit 12 may be disposed on one side or a plurality of sides of the contacted body 11.

  The detection unit 13 detects the contact portion of the contact object with the contacted body 11 over time. In addition, it can be said that the contact part of a contact object is the information represented by the contact area and the contact area of a contact object. The detection unit 13 detects, as an image, light scattered from the contacted body 11 (hereinafter, scattered light) as the contact object comes into contact with the contacted object 11 out of the light emitted from the light source unit 12. It is an optical instrument. The detection unit 13 is realized by a camera equipped with a semiconductor image sensor such as a charge coupled device (CCD) image sensor or a complementary metal oxide semiconductor (CMOS) image sensor, for example. .

  In addition, in FIG. 1, the example which has arrange | positioned the detection part 13 on the surface (back surface) on the opposite side to the surface where a contact thing contacts on the to-be-contacted body 11 is shown. However, the position where the detection unit 13 is arranged is not limited to the above position. The detection part 13 may be arrange | positioned at the surface (front surface) which a contact thing contacts.

  The applicator 15 is a writing instrument having a structure for applying the application 16 filled inside to the contacted body 11. The applicator 15 is realized using, for example, a felt pen container such as a whiteboard marker. However, the mode of the applicator 15 is not limited to the felt pen container.

  The application 16 is a substance that can be applied to the contacted body 11 or wiped off. The coating 16 contains a substance that scatters light in the wavelength region emitted from the light source unit 12 as a component. For example, when infrared light is used for the light source unit 12, the coating 16 is realized by a general whiteboard marker ink.

  The operation recognition unit 2 receives the scattered light image detected by the detection unit 13 and executes a process of detecting a touch operation. In the process of detecting the touch operation, the operation recognition unit 2 determines the presence / absence of the touch operation and the position where the touch operation is performed. Specifically, when the operation recognizing unit 2 receives the scattered light image indicating the contacted portion from the detection unit 13, whether or not the area of the contacting portion has changed (increased or decreased) over time. Judge. Further, when the area changes with time, the operation recognition unit 2 determines whether or not the contact position of the contact object is at a substantially constant position or whether or not the contact object has moved from the contact position. . Then, the operation recognition unit 2 determines that the touch operation has been performed when the contact area is changing and the position of the contact object is at a substantially constant position. Whether or not the position is substantially constant is determined, for example, by comparing with a predetermined threshold value. Then, the operation recognition unit 2 outputs information representing the determined result to the operation response unit 1.

  The operation recognition unit 2 is realized by, for example, a general personal computer (PC) on which an external device connection interface is mounted and a general-purpose operating system, driver software, and an image processing program are executed. More specifically, the operation recognition unit 2 is realized by a CPU of a computer that operates according to a program (recognition program). For example, the program may be stored in a personal computer or a storage unit (not shown) of the operation recognition device 10, and the CPU may read the program and operate as the operation recognition unit 2 according to the program.

  When the operation response unit 1 receives the result determined by the operation recognition unit 2, the operation response unit 1 makes a predetermined response according to the contact state with respect to the contacted body 11. For example, the operation response unit 1 makes a response according to the detected contact position. For example, the operation response unit 1 may display a pointer or a menu on the screen of a display device (not shown) according to the contact position.

  Next, the operation of the operation recognition device in this embodiment will be described. First, the principle by which the contact position and the contact area of the contact object are detected in the operation recognition device according to the present embodiment will be described. Next, a procedure for recognizing a touch operation based on detected information will be described.

  First, the light source unit 12 causes light outside the visible light region to enter from the side surface of the contacted body 11. FIG. 2 is an explanatory diagram illustrating an example of a state in which light from the light source unit 12 is incident on the contacted body 11. Assuming that the contact object is not in contact with the contacted body 11, the incident light is totally reflected on the surface of the contacted body 11 and is confined inside and guided as illustrated in FIG. 2. To wave.

  The detection unit 13 regularly captures an image of the contacted object 11 in the infrared region (for example, at 30 frames per second). Assuming that the contact object is not in contact with the contacted body 11, the image recorded by the detection unit 13 is a substantially black image. This is because the light emitted by the light source unit 12 is confined inside the contacted body 11 and does not reach the detection unit 13.

  Then, the case where the contact thing is contacting the to-be-contacted body 11 is demonstrated. As already described, typical contact objects include a coating 16 such as whiteboard marker ink, and a user's finger 14.

  FIG. 3 is an explanatory diagram illustrating an example of a light reflection state on the surface of the contacted body 11. As described above, the light emitted from the light source unit 12 is totally reflected on the surface of the contacted body 11. At this time, as illustrated in FIG. 3A, the light slightly oozes out from the contacted body 11 to the outside air side. In addition, the light of this exuded part is called an evanescent wave. When a contact object such as the coated object 16 or the user's finger 14 is in contact with the contacted object 11, the evanescent wave is optically coupled to the contact object. Then, as illustrated in B or C of FIG. 3, the evanescent wave is scattered outside the contacted body 11.

  Since the evanescent wave is scattered also on the back side of the contacted body 11 that is a transparent flat plate, a part of the light scattered on the back side reaches the detection unit 13 and the detection unit 13 detects the light. FIG. 4 is an explanatory diagram illustrating an example of an image frame detected by the detection unit 13. A portion of the shape of “I” that appears white and dark in the upper left of the image frame illustrated in FIG. 4 is a portion to which the coating material 16 is attached on the contacted body 11. In addition, an elliptical portion at the center of the image frame illustrated in FIG. 4 is a portion in which the user's finger 14 is in contact. In this way, the detection unit 13 observes the area where the contact object is in contact as an area having a specific shape.

  The operation recognition unit 2 receives the image taken by the detection unit 13 and calculates the position and contact area of the contact object. The position of the contact object is quantified as the coordinates of the region where the scattered light is detected in the image, and the contact area is quantified as the number of pixels included in this region. For example, the operation recognition unit 2 quantifies the contact position of the contact object as the coordinates of the region where the scattered light is detected in the image. In addition, the operation recognition unit 2 quantifies the contact area of the contact object as the number of pixels included in the contacted area.

  In the following description, the area (part) where the contact object is in contact is expressed as the position of the area. Here, the position of the area may be represented by the position of the center of gravity of the area, or may be represented by the center position (specifically, the average of the maximum value and the minimum value of each coordinate axis). The detection part 13 should just represent the position of each area | region based on the method defined beforehand.

  In other words, the expression of position with respect to a region having a finite spread in terms of surface is an ambiguous expression and includes ambiguity. For example, there is a possibility of indicating the center of gravity position of the region and a possibility of indicating the center position. In the operation recognition apparatus according to the present embodiment, if the definition of the position is consistent throughout the process, the effect does not change regardless of the definition method. Therefore, hereinafter, the area (part) where the contact object is in contact is expressed as the position of the area, unless otherwise distinguished.

  The above is the detection principle of the position and contact area of the contact object with respect to the contacted body. Next, a procedure for recognizing a touch operation from information on the position of the contact object and the contact area will be described. Here, before explaining a specific processing flow for recognizing a touch operation, the principle of detecting a touch operation by the operation recognition device in the present embodiment will be outlined.

  When categorized roughly in time series, the touch operation is (1) a phase in which a contact object such as a finger comes into contact with the contacted object (hereinafter referred to as “contact phase”), and (2) contact of the contact object with the contacted object. It includes three processes: a holding phase (hereinafter referred to as “holding phase”), and (3) a phase in which the contact object is released from the contacted object (hereinafter referred to as “leaving phase”).

  In the operation recognition device in the present embodiment, the operation recognition unit 2 determines the presence or absence of a touch operation by detecting the “contact phase” shown in (1) among the above three processes. Here, in the “contacting phase” of the touch operation, attention is paid to the characteristic that the area of the light scattering region with the contact of the contact object increases with time while the position of the region is substantially constant. That is, the operation recognition unit 2 determines the presence or absence of a touch operation based on the area and position of a region that changes with time.

  FIG. 5A is a graph illustrating an example of a contact area that changes with time when a touch operation is performed. FIG. 5B is a graph illustrating an example in which the distance between the contact position and the representative position (that is, the position of the region) is expressed in time series when the touch operation is performed.

  As illustrated in FIG. 5A, in the initial stage of the touch operation (ie, the “contacting phase”), the contact area with the contacted object increases from 0 to a finite value. In the holding stage (ie, “holding phase”), the contact area becomes substantially constant, and in the separation stage (ie, “release phase”), the contact area decreases to zero again.

  On the other hand, as illustrated in FIG. 5B, the position of the contact region hardly changes from the initial stage of the touch operation to the release stage. In the “contact phase” and “release phase”, the distance tends to be slightly larger. This is a phenomenon that is particularly characteristic of a touch operation with a finger, and usually represents a characteristic in which contact is made from the tip of the finger toward the center of the finger. However, the value is generally small and the position can be said to be almost constant.

  Hereinafter, a specific processing flow for recognizing a touch operation based on the above-described principle will be described. In the operation recognition device according to this embodiment, the operation recognition unit 2 recognizes the touch operation.

  FIG. 6 is a flowchart illustrating an example of processing performed by the operation recognition unit 2. First, the operation recognizing unit 2 receives image frames F1 and F2 continuous on the time axis from the detecting unit 13 (steps S1 and S2). Specifically, the operation recognition unit 2 determines whether the image frame F1 is received from the detection unit 13 (step S1). When the image frame F1 is not received (N in step S1), the operation recognition unit 2 repeats the process of step S1 for receiving the image frame. On the other hand, when the image frame F1 is received (Y in step S1), the operation recognition unit 2 determines whether the image frame F2 is received from the detection unit 13 (step S2). When the image frame F2 is not received (N in step S2), the operation recognition unit 2 repeats the process of step S2 for receiving the image frame.

  When the image frames F1 and F2 are received (Y in step S2), the operation recognition unit 2 calculates the difference image D1 = F2-F1 (step S3). Note that when the operation recognition unit 2 calculates the image difference, the difference value is replaced with 0 for a pixel having a negative difference value. By doing in this way, only the area | region where the contact area increased can be extracted. For example, the operation recognition unit 2 may calculate a difference image by comparing pixel values in an image frame.

  FIG. 7 is an explanatory diagram illustrating an example of an image frame at each time point. In the example illustrated in FIG. 7, it is detected that the image frames f1 to f5 are detected in time series. Here, it is assumed that no contact object is in contact at the time when the image frame F1 is detected, and that a touch operation with a finger is started at the time when the image frame F2 is detected.

  In this case, the image frame F1 is a black image like the image frame f1 illustrated in FIG. 7A, and the image frame F2 is the image frame f2 illustrated in FIG. 7B. Further, the difference image D1 is also an image frame f2 illustrated in FIG. Here, in the image frame f2, the area on the white ellipse corresponds to the position where the finger is in contact.

  Subsequently, the operation recognizing unit 2 identifies a region independent from the portion where the scattered light is detected based on the difference image D1 (step S4). When the difference image D1 is the image frame f2 illustrated in FIG. 7B, the area is only one elliptical area. Therefore, the operation recognition unit 2 extracts this area.

As a specific method for identifying independent regions, a processing method called labeling generally known in the field of image processing can be used. The details of the labeling process are described in, for example, Reference Document 1 below, and will not be described here.
[Reference 1]
Nara Institute of Science and Technology OpenCV Programming Book Creation Team, “OpenCV Programming Book”, Mainichi Communications, September 2007, p. 201-204

  Subsequently, the operation recognition unit 2 calculates the position of each identified area (step S5). Here, the operation recognition part 2 shall calculate the coordinate of the center position of an area | region as a position of each area | region. Then, the operation recognizing unit 2 uses the calculated position coordinates (Xi, Yi) (where i = 1 to N; N is the number of identified areas. In this example, N = 1) for subsequent processing. The data is stored in a memory (not shown) (step S6).

  FIG. 8 is an explanatory diagram illustrating an example of a difference image calculated from an image frame. Note that the dotted lines illustrated in FIG. 8 indicate the coordinates calculated by the operation recognition unit 2. For example, the operation recognition unit 2 calculates the position of the region from the difference image illustrated in FIG.

  Subsequently, the operation recognition unit 2 receives an image frame F3 subsequent to the image frame F2 from the detection unit 13. Specifically, the operation recognition unit 2 determines whether or not the image frame F3 has been received (step S7). When the image frame F3 is not received (N in step S7), the operation recognition unit 2 repeats the process of step S7 for receiving the image frame. On the other hand, when the image frame F3 is received (Y in step S7), a difference image D2 = F3-F2 between the image frame F2 and the image frame F3 is calculated (step S8). Here, at the time illustrated in FIG. 7C, it is assumed that the touch operation with the finger further proceeds and the image of the image frame f3 is detected. At this time, the differential image D2 is an image including a donut-shaped region as illustrated in FIG. 8B.

  As in the case of the difference image D1 shown in steps S4 to S5, the operation recognition unit 2 executes an independent area identification process (step S9) and a position calculation process for each area for the difference image D2 (step S9). S10). Here, the position coordinates (Aj, Bj) (where j = 1 to M; M is the number of identified areas. In this example, M = 1) are calculated. Specifically, the operation recognition unit 2 calculates the coordinates indicated by the dotted line illustrated in FIG.

  Thereafter, the operation recognition unit 2 compares the coordinates (Xi, Yi) with the coordinates (Aj, Bj). If there is a combination of coordinates whose mutual distance is equal to or less than a certain threshold, the operation recognition unit 2 determines that a touch operation has been performed (step S11). Note that, for example, a value such as 10 pixels may be obtained experimentally in advance as the fixed threshold value. Hereinafter, the determination by the operation recognizing unit 2 that a touch operation has been performed is referred to as a determination that there is a touch operation.

  In addition, when it is determined that there is a touch operation, the operation recognition unit 2 may use any of the coordinates (Xi, Yi) or (Aj, Bj) as the contact position. In addition, the operation recognition unit 2 may use a value derived from both coordinates such as an average value of the coordinates (Xi, Yi) and the coordinates (Aj, Bj) as the contact position.

  The operation recognition unit 2 outputs the touch operation determination result and the derived contact position to the operation response unit 1. Specifically, when the operation recognition unit 2 determines that there is a touch operation, the operation recognition unit 2 outputs information indicating that the operation is different from the application operation of the applied material to the operation response unit 1. In addition, according to the said process, it can also determine with touch operation in several independent area | regions. In this case, the operation recognition unit 2 transmits a plurality of corresponding contact positions to the operation response unit 1, so that even when a plurality of touch operations are performed at the same time, the operation response unit 1 individually sets each contact position. It becomes possible to grasp.

  Note that when the touch operation is not performed, the operation recognition unit 2 may not output the determination result to the operation response unit 1. Further, the information output by the operation recognition unit 2 is not limited to information related to a touch operation such as a determination result or a contact position. For example, when the contact of the contact object is detected by a method other than the touch operation, such as when the application object is applied, the operation recognition unit 2 may output information indicating the contact state of the contact object to the operation response unit 1. Good. More specifically, the information representing the contact state of the contact object other than the touch operation includes image information received from the detection unit 13 and image information obtained by partially processing the received image information. The partially processed image information includes, for example, image information in which a region where a touch operation has been performed on the image information received from the detection unit 13 is filled with a background color. By transmitting such information, the operation response unit 1 can perform different processes by distinguishing between the touch operation and the application operation.

  Finally, the operation recognition unit 2 stores the image frame F3 by replacing it with the image frame F2. Furthermore, the operation recognizing unit 2 replaces the coordinates (Aj, Bj) with the coordinates (Xi, Yi) and stores them (step S12). Thereafter, the operation recognizing unit 2 repeats the processes after step S7 for receiving a new image frame.

  Note that, by performing the repeated processing, there is a possibility that the determination of the presence of the touch operation may be made a plurality of times for a single touch operation. Therefore, the operation recognition unit 2 performs a plurality of touch determinations by combining the determination of the presence of touch operations on a plurality of floors within a certain time (for example, 100 ms) into the determination of the presence of a single touch operation. Can be avoided.

  When a touch operation is recognized based on such a method, for example, scattered light from a substance that is constantly attached to the contacted object (applied object) such as the applied object 16 appears on the difference image. Absent. Accordingly, it is possible to avoid such a coating 16 being recognized as a touch operation. Further, in the process in which the application 16 is applied by the applicator 15, the contact area of the application 16 increases. That is, the applied material 16 at the time of application appears on the difference image. However, since the position where the application 16 is applied changes in the moving direction of the applicator 15, it can be avoided that it can be recognized as a touch operation. Hereinafter, these contents will be further described.

  FIG. 9 is an explanatory diagram illustrating an example of a relationship between an image frame and a difference image. In the example shown in FIG. 9, the difference image d1 is an image representing the difference between the image frames f7 and f6, and the difference image d2 is an image representing the difference between the image frames f8 and f7.

  Similar to the description of the above operation, the image frame F1 is a black image as illustrated in the image frame f6 of FIG. 9A at the start of application, and the image frame F2 immediately after the start of application is illustrated in FIG. 9B. It is assumed that the image is exemplified in the image frame f7. At this time, the difference image D1 is an image like the difference image d1 illustrated in FIG. Further, a position indicated by a dotted line in the difference image d1 is calculated as the position of the region extracted from the difference image d1.

  Thereafter, drawing proceeds, and it is assumed that the detection unit 13 detects an image frame f8 illustrated in FIG. 9C as the image frame F3. At this time, the difference image D2 is an image like the difference image d2 illustrated in FIG. Further, a position indicated by a dotted line in the difference image d2 is calculated as the position of the region extracted from the difference image d2.

  As can be seen from this, in the application operation of the application 16, the position of the application 16 applied in accordance with the drawing shifts in the moving direction of the applicator 15. Therefore, by appropriately setting a threshold value for determining whether or not it has moved, detection of a touch operation in the process of applying the application 16 can be avoided.

  In addition, for example, it is assumed that the user has drawn an oval figure that is filled inside. Even in this case, the detection unit 13 can detect the drawing as a line segment by appropriately setting an interval for detecting the image frame. Therefore, it can be avoided that the operation recognition unit 2 determines such a drawing process as a touch operation. The image frame detection interval may be set to 30 frames per second, for example.

  Furthermore, when the drawing process using the applicator 15 and the touch operation are performed at the same time, the presence of the touch operation is determined only at the position where the touch operation is performed. This is clear from the above description.

  As described above, according to the present embodiment, the detection unit 13 detects the contact portion of the contact object with the contacted body 11 with the passage of time. Then, when the area of the contact area changes with time, the operation recognition unit 2 recognizes an event (touch operation) in which the change in the position of the area is within a predetermined distance (that is, a substantially constant position). To do. Furthermore, when the area of the contact region changes with time, the operation recognition unit 2 also recognizes an event in which the change in the position of the region exceeds a predetermined distance (an event in which the touched position moves). When it is recognized that the detected contact operation is a touch operation, the operation recognition unit 2 displays information indicating the touch operation, which is different from the information indicating the application operation of the application. Output to the response unit 1. Therefore, the recognition accuracy of the operation on the contacted object can be improved.

  That is, by using the operation recognition device according to the present embodiment, it is possible to identify the touch operation because the recognition accuracy of the touch operation can be improved even when there is a sensor input caused by a cause other than the touch operation. become.

  In the above processing, a region where the contact area increases by calculating the image difference is extracted, and the position of the calculated difference region is extracted as an event that is almost constant with the position of the contact region. That is, in the above description, the position of the contact area is not directly calculated. However, it can be said that the above process is substantially equivalent to extracting a contact region where the contact position is substantially constant and the contact area is increased.

  For example, by performing a labeling process on each image frame without calculating a difference image, and performing a position calculation on each area obtained by this, an area where the contact position is almost constant while changing the contact area is obtained. It is possible to detect. However, in this case, the number of areas extracted by the labeling process increases, and the processing time of the labeling process itself and the position calculation process increases. However, in the present embodiment, it is possible to suppress an increase in processing time by calculating the difference image.

In the above description, among the three processes included in the touch operation (1) “contact phase”, (2) “hold phase”, and (3) “release phase”, the operation recognition unit 2 The operation of detecting the “contact phase” shown in (1) and determining that a touch operation has been performed when this phase is detected has been described. As an application of this, the operation recognition unit 2 detects (2) “hold phase” and (3) “release phase” after detecting the “contact phase”, and outputs the information to the operation response unit 1. It is also possible. As a method for the operation recognition unit 2 to determine contact in (2) “holding phase” and (3) “release phase”, for example, an object tracking method generally known in the image processing region is used. do it. Specifically, the operation recognizing unit 2 may start the tracking of the area at the contact position when detecting the “contacting phase”, and determine the separation event when the tracking target area disappears. . When such a method is used, the position where contact is detected and the position where separation is detected need not be the same. It is also easy to identify trajectory information (drag information) on which the contact object moves in the contact holding state. Note that details of the object tracking process are described in Reference Document 2 below, for example, and thus the description thereof is omitted here.
[Reference 2]
Nara Institute of Science and Technology OpenCV Programming Book Creation Team, “OpenCV Programming Book”, Mainichi Communications, September 2007, p. 218-237

  Further, (3) as a method of detecting the touch operation by the “release phase” alone, the operation recognition unit 2 may use a method of extracting a region where the contact area decreases with time at a substantially constant position. In order to extract a region where the contact area decreases with time, the sign of the image difference result calculated in (1) “contacting phase” may be reversed.

  Further, in the present embodiment, an example in which an FTIR optical touch panel is used as an apparatus capable of detecting the contact position and contact area of a contact object with respect to the contacted body 11 has been described. However, the device for detecting the contact position and the contact area is not limited to the above device. As long as the device can detect the contact position and the contact area, the operation recognition device according to the present embodiment may be configured using another device.

  Further, the determination of whether or not the contact area is increased and the position is a substantially constant contact area may be performed by arranging different types of operation recognition devices in a stacked manner. For example, the FTIR touch panel and the capacitive touch panel may be stacked. The reason why the capacitive touch panels are stacked is that by adjusting the sensitivity, it is possible to identify the presence of a finger or the like in the vicinity without touching the touch panel itself. By arranging in this way, while the FTIR type touch panel detects the adhesion state of the applied material to the contacted body 11, the touch operation to the contacted body 11 is performed by a capacitive touch panel sensor. It becomes possible to detect

Embodiment 2. FIG.
Next, an information processing system according to the present invention will be described. The information processing system according to the present invention includes the operation recognition device exemplified in the first embodiment, and utilizes a sensor input generated by a cause other than the touch operation as input information different from the touch operation. .

  FIG. 10 is a block diagram showing an embodiment of an information processing system according to the present invention. In addition, about the structure similar to the operation recognition apparatus illustrated in 1st Embodiment, the code | symbol same as FIG. 1 is attached | subjected and description is abbreviate | omitted.

  The information processing system 20 in the present embodiment includes a display unit 21 and an operation response unit 1 in addition to the contacted object 11, the light source unit 12, the detection unit 13, and the operation recognition unit 2. The display unit 21 includes a screen 22 and a projector 23. That is, the information processing system according to the present embodiment includes the display unit 21 including the screen 22 and the projector 23 and the operation response unit 1 in addition to the operation recognition device 10 according to the first embodiment.

  As described in the first embodiment, the operation recognition device (more specifically, the contacted body 11, the light source unit 12, the detection unit 13, and the operation recognition unit 2) The contact is monitored, and image information captured by the detection unit 13 or image information obtained by partially processing the image information is output to the operation response unit 1 together with information related to the touch operation.

  The operation response unit 1 performs processing according to the position where the contact operation is detected based on the event recognized by the operation recognition device (more specifically, the operation recognition unit 2). For example, when receiving from the operation recognition unit 2 an image of scattered light captured by the detection unit 13 and information on the presence / absence of a touch operation and the position at which the touch operation is performed, the operation response unit 1 determines the contact position based on each event. The display unit 21 is instructed to perform display processing according to the above. Specific contents instructed by the operation response unit 1 will be described later. In the following description, information indicating an event related to contact, such as contact with a contact object or separation of a contact object from a contacted object, is referred to as an event type.

  Similar to the operation recognition unit 2, the operation response unit 1 is realized by a general personal computer. More specifically, the operation response unit 1 is realized by a CPU of a computer that operates according to a program (recognition program). For example, the program may be stored in a storage unit (not shown) such as a personal computer, and the CPU may read the program and operate as the operation response unit 1 according to the program. Moreover, the operation response part 1 and the operation recognition part 2 may be implement | achieved by single hardware (for example, personal computer).

  The display unit 21 displays contents according to instructions from the operation response unit 1. The screen 22 is disposed on the back surface of the contacted object 11, and the projector 23 displays an image of scattered light, a pointer and a menu corresponding to the touch operation, and the like on the screen 22 according to an instruction from the operation response unit 1. To display. The displayed content of the screen 22 is recognized by the user through the contacted object 11 which is a transparent flat plate.

  In the present embodiment, the case where the display unit 21 includes the screen 22 and the projector 23 will be described. However, the content of the display unit 21 is not limited to the screen 22 and the projector 23. If the display device is capable of displaying digital information, the display unit 21 may be a flat panel display such as a liquid crystal display, a plasma display, or a field emission display, for example.

  Next, the operation of the information processing system in this embodiment will be described. In the following description, an example in which the operation response unit 1 receives either image information or information related to a touch operation from the operation recognition unit 2 and performs processing based on the received information will be described as an example. Here, the information received as the image information includes information indicating that a coating application operation is performed.

  FIG. 11 is a flowchart illustrating an example of processing performed by the operation response unit 1. FIG. 12 is an explanatory diagram illustrating an example of an operation on the contacted body 11 and a display on the display unit 21. The operation response unit 1 receives either image information or information related to the touch operation from the operation recognition unit 2 (step S21). When the received information is image information (“image information” in step S21), the operation response unit 1 displays the size of this image (eg, 640 × 480 pixels) on the display unit 21 (more specifically, the screen 22). Scale up or down to fit size (eg 1024 x 768 pixels). Further, the operation response unit 1 performs coordinate conversion so that the physical position where the contact of the contact object is detected on the contacted body 11 coincides with the position on the display unit 21 (more specifically, the screen 22). (Step S22). Then, the operation response unit 1 instructs the display unit 21 to display an enlarged or reduced image (step S23). Here, the operation response unit 1 may instruct the display unit 21 to perform display processing after giving some ideas on the expression such as reversing the brightness of the image. Thereafter, the operation response unit 1 repeats the processing after step 21 for determining the contents of data reception.

  In step S22, information necessary for coordinate conversion is obtained in advance by performing a calibration process between the operation recognition device (more specifically, the operation recognition unit 2) and the display unit 21. be able to. About this method, the method currently used with the general touch panel should just be used.

  For example, it is assumed that the graphic illustrated in FIG. 12A is drawn on the contacted body 11 using the applicator 15. When the operation response unit 1 receives the graphic image from the operation recognition unit 2, the operation response unit 1 displays it in accordance with the position on the display unit 21 (more specifically, the screen 22). As a result, the figure drawn by the coating 16 is in a state of being lit up from the back. At this time, the visual feedback lit up from the back serves as a clue to the user that the drawn figure is surely digitized. By performing the processing as described above, the information processing system of the present invention can be used as a system such as a digital whiteboard.

  At this time, the operation response unit 1 may display the position on the screen 22 corresponding to the position where the contact object such as the application object 16 is in contact with a color different from other display positions. The display may be displayed with a different brightness from other display positions. By doing in this way, touch operation and contact other than touch operation can be distinguished.

  On the other hand, when the information received by the operation response unit 1 in step S21 is information relating to the touch operation such as information indicating the presence / absence of the touch operation and the position thereof (“touch information” in step S21), the operation response The unit 1 executes appropriate processing according to the event type (specifically, a contact event, a leaving event, etc.) and the position where the contact is detected (step S24). The operation response unit 1 may execute, for example, pointer display processing at a position where a touch operation is detected, menu display processing, selection processing for an object such as an icon displayed at a position where a touch operation is detected, or the like. Good. However, the process performed by the operation response unit 1 is not limited to these processes. Thereafter, the operation response unit 1 repeats the processing after step 21 for determining the contents of data reception.

  As described above, when the operation response unit 1 determines that a touch operation has been performed, for example, the operation response unit 1 performs a process different from that performed when a coating is applied. Specifically, the operation response unit 1 may perform processing with visually different contents when a touch operation is performed and when a coating is applied. Hereinafter, a specific example of processing performed by the operation response unit 1 when information related to a touch operation is received will be described with reference to FIG.

  The operation response part 1 will display a menu in the position where the touch operation was detected, if the information regarding a touch operation is received in the state illustrated to Fig.12 (a) (refer FIG.12 (b)). Subsequently, it is assumed that the user selects a save command from the menu displayed at the touched position (see FIG. 12C). Then, when the operation recognition unit 2 detects a touch operation on the upper left position of the figure and transmits touch information to the operation response unit 1, the operation response unit 1 instructs the display unit 21 to display a pointer at the position. (See FIG. 12D). Furthermore, from the state of FIG. 12D, when the operation recognition unit 2 detects a touch operation on the lower right position of the figure and transmits touch information to the operation response unit 1, the operation response unit 1 detects the touched position. A rectangular area is defined and a new menu is displayed on the display unit 21 (see FIG. 12E). In this state, when the user performs a touch operation at the position of the OK command in the menu, the operation response unit 1 executes the command (OK command) (see FIG. 12F).

  In the above, the process which the operation response part 1 performs according to touch operation was demonstrated. However, the process which the operation response part 1 performs according to the information regarding touch operation is not limited to the said content. For example, when the operation recognition unit 2 detects a plurality of touches at the same time or detects a drag state, the operation response unit 1 may perform an appropriate process according to the detected state.

  In the above description, the case where the information received by the operation response unit 1 from the operation recognition unit 2 is one of image information and information related to a touch operation has been described. However, the operation response unit 1 may receive image information and information related to the touch operation from the operation recognition unit 2 at the same time. In this case, the process in each case may be performed according to the received content.

  As described above, according to the present embodiment, since the operation recognition unit 2 can identify a touch operation, the operation response unit 1 can input a sensor input generated by a cause other than the touch operation, separately from the touch operation. Can be used as information. As a result, an information processing apparatus (system) that can input various patterns can be realized with a single operation recognition apparatus.

  That is, in a general device, when a sensor input other than the touch operation is detected, it is difficult to identify the touch operation. Further, in a general apparatus, it is difficult to use information other than the touch operation that can be sensed by the sensor as input information different from the touch operation. However, according to the present embodiment, the operation recognition unit 2 identifies the touch operation, and notifies the operation response unit 1 of information other than the operation together with the identified information. Therefore, sensor input caused by a cause other than the touch operation can be used as input information different from the touch operation.

  Next, the minimum configuration of the present invention will be described. FIG. 13 is a block diagram showing an example of the minimum configuration of the operation recognition apparatus according to the present invention. FIG. 14 is a block diagram showing the minimum configuration of the information processing system according to the present invention. The operation recognition device according to the present invention is a contact object 80 (for example, a user's finger 14) contacted by an operator or a contacted object 80 (for example, a coated object 16) to be contacted by a contact object (for example, a coated object 16) applied by the operator. The contact position detection means 81 (for example, constantly) detects the contact position of the contact object 11) and the contact position and contact area (for example, a contacted portion) of the contact object with time (for example, regularly). In the case where the area of the contact area increases or decreases with the passage of time, the change of the area position (for example, the position of the area) representing the position of the contact area is within a predetermined distance (for example, An event that is a substantially constant position) is recognized as a first event (for example, a touch operation), and an event in which the change in the region position exceeds the distance (for example, the contact has moved) is a second event (for example, a touch operation). Recognizing means 82 (for example, the operation recognizing unit 2) for recognizing that the application 16 is applied), information indicating the recognized event (for example, determination result of touch operation, image information used in determination) Event output means 83 for outputting to the device (for example, the operation response unit 1).

  When the first event is recognized, the event output unit 83 is information indicating the first event (for example, the event recognition result or the image information detected by the detection unit 13), Information representing an event different from the event (for example, image information on which the coating 16 is applied) is output to another device.

  With such a configuration, the recognition accuracy of the operation on the contacted object can be improved. Specifically, the recognition accuracy of the touch operation can be improved.

  Further, the contact position detecting means 81 may detect the contact area of the contact object with respect to the contacted object by measuring light scattered by the contact object (for example, scattered light, evanescent wave).

  Further, the contacted body has a structure capable of guiding light (for example, infrared light) inside, and light is incident on the inside of the contacted body (for example, light is incident by the light source unit 12). The contact position detecting means 81 may detect the contact area of the contact object with respect to the contacted object by measuring light scattered when the contact object comes into contact with the contacted object.

  Further, the contact position detecting means 81 may detect contact with a part of the human body (for example, the user's finger 14), and detect contact with a substance (for example, the coated material 16) that can be applied to the contacted body. May be.

  In addition, when the event output unit 83 detects contact of a contact object due to an event other than the first event (for example, movement processing while the contact object is kept in contact or application state of the application object), the contact of the contact object is detected. Information indicating the state may be output to another device. By doing in this way, it becomes possible to perform the process which distinguishes a contact position and the position where there was no contact.

  The information processing system according to the present invention includes a contacted object 80, a contact position detecting unit 81, a recognizing unit 82, and a response unit 84 that makes a predetermined response according to the recognized event. Yes. When the first event (for example, touch operation) is recognized, the response means 84 is a response (for example, a pointer or menu display) determined according to the first event, and the second event The response different from the response (for example, displayed in a color or brightness different from other display positions) is performed. The contacted body 80, the contact position detecting means 81, and the recognizing means 82 are the same as the configuration of the operation recognizing device according to the present invention.

  Even with such a configuration, the recognition accuracy of the operation on the contacted object can be improved. Specifically, the recognition accuracy of the touch operation can be improved.

  A part or all of the above-described embodiment can be described as in the following supplementary notes, but is not limited thereto.

(Supplementary note 1) A contact to be contacted by a contact object to be contacted by an operator or a contact object applied by the operator, and a contact for detecting a contact position and a contact area of the contact object with respect to the contact object with time. In the case where the area of the contact area increases or decreases with the passage of time, an event in which a change in the area position representing the position of the contact area is within a predetermined distance is recognized as a first event. Recognizing means for recognizing an event in which the change in the region position exceeds the distance as a second event, and event output means for outputting information representing the recognized event to another device, The output means outputs, when the first event is recognized, information indicating the first event, which is different from the information indicating the second event, to another device. Characterizing operation Apparatus.

(Additional remark 2) The operation recognition apparatus of Additional remark 1 which a contact position detection means detects the contact area | region of the contact object with respect to a to-be-contacted object by measuring the light scattered by a contact object.

(Appendix 3) The contacted body has a structure capable of guiding light inside, light is incident on the inside of the contacted body, and the contact position detecting means is in contact with the contacted body. The operation recognizing device according to appendix 1 or appendix 2, wherein the contact area of the contact object with respect to the contacted object is detected by measuring light scattered as the object comes into contact.

(Supplementary note 4) The operation recognition device according to any one of Supplementary note 1 to Supplementary note 3, wherein the contact position detecting means detects contact by a part of a human body.

(Supplementary Note 5) The operation recognition device according to any one of Supplementary Note 1 to Supplementary Note 4, wherein the contact position detecting unit detects contact of a substance that can be applied to the contacted body.

(Appendix 6) When the event output means detects contact of the contact object due to an event different from the first event, the event output means outputs information indicating the contact state of the contact object to other devices. The operation recognition apparatus as described in any one of them.

(Appendix 7) A contact object that is contacted by a contact object to be contacted by an operator or a contact object that is applied by the operator, and a contact that detects a contact position and a contact area of the contact object with respect to the contact object with time. In the case where the area of the contact area increases or decreases with the passage of time, an event in which a change in the area position representing the position of the contact area is within a predetermined distance is recognized as a first event. Recognizing means for recognizing an event in which the change in the region position exceeds the distance as a second event, and responding means for performing a predetermined response in accordance with the recognized event, When the first event is recognized, the response is determined according to the first event and is different from the response determined according to the second event. Characteristic features Processing system.

(Supplementary note 8) The information processing system according to supplementary note 7, wherein the contact position detection unit detects a contact area of the contact object with respect to the contacted object by measuring light scattered by the contact object.

(Supplementary Note 9) The contacted body has a structure capable of guiding light inside, and light is incident on the inside of the contacted body, and the contact position detecting means is in contact with the contacted body. The information processing system according to appendix 7 or appendix 8, wherein the contact area of the contact object with respect to the contacted object is detected by measuring light scattered as the object contacts.

(Supplementary note 10) The information processing system according to any one of supplementary note 7 to supplementary note 9, wherein the contact position detecting means detects contact by a part of a human body.

(Supplementary note 11) The information processing system according to any one of supplementary note 7 to supplementary note 10, wherein the contact position detecting means detects contact of a substance that can be applied to the contacted body.

(Supplementary Note 12) The response means includes display means for displaying information corresponding to the recognized event at a corresponding position of the contacted object, and the response means is contact of the contact object by an event different from the first event. When the position is detected, a process for causing a change visually different from the response determined according to the first event is performed on the position on the display unit corresponding to the position touched by the contact object 7 To 11. The information processing system according to any one of appendix 11.

(Supplementary note 13) The response determined according to the first event includes processing for displaying a pointer at the position of the display means corresponding to the contact position where the first event is detected. The information processing system as described in any one of them.

(Supplementary Note 14) The response determined according to the first event includes a process of displaying a menu at the position of the display unit corresponding to the contact position where the first event is detected. The information processing system as described in any one of them.

(Supplementary Note 15) The response determined according to the first event includes processing indicating that the object displayed at the position of the display unit corresponding to the contact position where the first event is detected is selected. The information processing system according to any one of appendix 7 to appendix 14.

(Supplementary Note 16) The response means displays the position on the display means corresponding to the position touched by the contact object in a color different from the other display positions, thereby determining the response determined according to the first event. Is the information processing system according to any one of supplementary notes 12 to 15, which causes a visually different change.

(Supplementary Note 17) The response means displays the position on the display means corresponding to the position touched by the contact object with a lightness different from that of the other display positions, thereby determining the response determined according to the first event. Is the information processing system according to any one of supplementary notes 12 to 16, which causes a visually different change.

(Additional remark 18) With respect to the contacted object contacted by the contact object which an operator contacts, or the contact object apply | coated by an operator, the contact position and the contact area of the said contact object which contact the said contact object with progress of time Detecting, when the area of the contact area increases or decreases over time, recognizes an event in which a change in area position representing the position of the contact area is within a predetermined distance as a first event, When an event in which the change in the region position exceeds the distance is recognized as a second event, information representing the recognized event is output to another device, and information representing the event is output to another device When the first event is recognized, information indicating the first event and information different from the information indicating the second event is output to another device, Operation recognition method to do.

(Additional remark 19) The operation recognition method of Additional remark 18 which detects the contact area | region of the contact object with respect to a to-be-contacted object by measuring the light scattered by a contact object.

(Supplementary note 20) The contact position and the contact area of the contact object with the contact object contacted by the contact object contacted by the operator or the contact object applied by the operator and the contact area of the contact object over time Detecting, when the area of the contact area increases or decreases over time, recognizes an event in which a change in area position representing the position of the contact area is within a predetermined distance as a first event, An event in which the change in the region position exceeds the distance is recognized as a second event, a predetermined response is made according to the recognized event, and when the response is made, the first event is recognized If so, an information processing method characterized in that a response determined according to the first event and a response different from the response determined according to the second event is performed.

(Supplementary note 21) The operation recognition method according to supplementary note 20, wherein the contact area of the contact object with respect to the contacted object is detected by measuring light scattered by the contact object.

(Additional remark 22) The operation recognition program applied to the computer provided with the to-be-contacted object contacted by the contact thing which an operator contacts, or the contact thing apply | coated by the operator, Comprising: The said to-be-contacted object is applied to the said computer Contact position detection processing for detecting the contact position and contact area of the contact object with respect to time, and when the area of the contact area increases or decreases with time, the change of the area position representing the position of the contact area is changed Recognition processing for recognizing an event within a predetermined distance as a first event, recognizing an event in which the change in the region position exceeds the distance as a second event, and information representing the recognized event If the first event is recognized in the event output process, information indicating the first event is output. The information different from the second information indicating an event, operation recognition program for output to another device.

(Additional remark 23) The operation recognition program of Additional remark 22 which detects the contact area | region of the contact object with respect to a to-be-contacted object by measuring the light scattered by a contact object in a contact position detection process.

  The present invention is preferably applied to an operation recognition device that can recognize a touch operation.

DESCRIPTION OF SYMBOLS 1 Operation response part 2 Operation recognition part 10 Operation recognition apparatus 11 Contact object 12 Light source part 13 Detection part 14 User's finger | toe 15 Application tool 16 Application object 20 Information processing system 21 Display part 22 Screen 23 Projector f1-f7 Image frame d1 , D2 Difference image

Claims (10)

A contacted object that is contacted by a contact object that the operator contacts or a contact object that is applied by the operator;
A contact position detecting means for detecting a contact position and a contact area of a contact object with respect to the contacted object over time;
When the area of the contact region increases or decreases over time, an event in which a change in the region position representing the position of the contact region is within a predetermined distance is recognized as a first event, and the region position Recognizing means for recognizing an event in which the change in the distance exceeds the distance as a second event;
An event output means for outputting information representing the recognized event to another device;
The event output means outputs information indicating the first event, which is different from the information indicating the second event, to another device when the first event is recognized. An operation recognition device characterized by that. The operation recognition apparatus according to claim 1, wherein the contact position detection unit detects a contact area of the contact object with respect to the contacted object by measuring light scattered by the contact object. The contacted body has a structure capable of guiding light inside, and light is incident on the inside of the contacted body.
The contact position detecting means detects a contact area of the contact object with respect to the contacted object by measuring light scattered when the contact object comes into contact with the contacted object. The operation recognition device described. The operation recognition apparatus according to any one of claims 1 to 3, wherein the contact position detection unit detects contact by a part of a human body. The operation recognition apparatus according to any one of claims 1 to 4, wherein the contact position detection means detects contact of a substance that can be applied to the contacted body. The event output means outputs information indicating a contact state of the contact object to another device when the contact of the contact object due to an event different from the first event is detected. The operation recognition apparatus of any one of Claims. A contacted object that is contacted by a contact object that the operator contacts or a contact object that is applied by the operator;
A contact position detecting means for detecting a contact position and a contact area of a contact object with respect to the contacted object over time;
When the area of the contact region increases or decreases over time, an event in which a change in the region position representing the position of the contact region is within a predetermined distance is recognized as a first event, and the region position Recognizing means for recognizing an event in which the change of the distance exceeds the distance as a second event;
Response means for performing a predetermined response according to the recognized event,
When the first event is recognized, the response means is a response determined according to the first event, and is different from the response determined according to the second event. An information processing system characterized by doing. Detecting the contact position of the contact object and the contact area of the contact object over time with respect to the contact object contacted by the contact object contacted by the operator or the contact object applied by the operator, and the contact area of the contact object over time,
When the area of the contact region increases or decreases over time, an event in which a change in the region position representing the position of the contact region is within a predetermined distance is recognized as a first event, and the region position An event where the change in the distance exceeds the distance is recognized as a second event,
Output information representing the recognized event to another device,
When the information representing the event is output to another device, if the first event is recognized, the information indicating the first event is the information indicating the second event. An operation recognition method characterized by outputting different information to another device. Detecting the contact position of the contact object and the contact area of the contact object over time with respect to the contact object contacted by the contact object contacted by the operator or the contact object applied by the operator, and the contact area of the contact object over time,
When the area of the contact region increases or decreases over time, an event in which a change in the region position representing the position of the contact region is within a predetermined distance is recognized as a first event, and the region position An event where the change in the distance exceeds the distance is recognized as a second event,
Depending on the recognized event, perform a predetermined response,
When performing the response, if the first event is recognized, the response is determined according to the first event and is different from the response determined according to the second event. An information processing method characterized by performing a response. An operation recognition program applied to a computer provided with a contacted object to be contacted by a contact object to be contacted by an operator or a contact object applied by the operator,
In the computer,
A contact position detection process for detecting a contact position and a contact area of a contact object with respect to the contacted object over time;
When the area of the contact region increases or decreases over time, an event in which a change in the region position representing the position of the contact region is within a predetermined distance is recognized as a first event, and the region position A recognition process for recognizing an event in which the change in the distance exceeds the distance as a second event; and
Causing event output processing to output information representing the recognized event to another device;
When the first event is recognized in the event output process, information indicating the first event and information different from the information indicating the second event is sent to another device. Operation recognition program for output.

Images