KR20170068679A

KR20170068679A - A method of estimating horizontal or vertical force on the surface of a interface device by processing of markers’ image installed on the device

Info

Publication number: KR20170068679A
Application number: KR1020150175127A
Authority: KR
Inventors: 이수웅; 권순오; 이강원
Original assignee: 한국생산기술연구원
Priority date: 2015-12-09
Filing date: 2015-12-09
Publication date: 2017-06-20
Also published as: KR101767490B1

Abstract

The present invention relates to a method for estimating a force in a vertical direction and a horizontal direction using image processing of a plurality of markers provided on a cushion or a sheet, and more particularly, A method of recording and processing a change in a changing marker image by an image pickup unit and using a cushion made of a soft material is advantageous in that the quality of sensibility such as touch can be increased and it can be realized at low cost .

Description

A method of estimating a force in a vertical direction and a horizontal direction using image processing of a marker provided in an interface device. {A method of estimating horizontal or vertical force on the surface of a device by processing of markers' image installed on the device}

A force sensor is a device for measuring a force or pressure of one or three dimensions applied from the outside. The force sensor receives an external force, converts the force into an electric signal of a corresponding magnitude, and outputs the electric signal.

Since the commonly used forceps are installed and used with other mechanical elements, the possibility of contact with a human user is rare. However, in recent years, healthcare technology has been developed, and at the point where the demand of a service robot or the like performing a direct request of a person is spreading, a technology capable of accepting an input operated by a human user and finely inputting Is required.

Conventionally, in order to meet such a need, a contact made of a warm and soft material has been applied to the contact portion in order to increase the contact feeling of the portion contacting with the user and to reduce the cold feeling. In this case, There is a problem that an error occurs in the measurement of the user input.

Korean Patent Publication No. 10-1169940 (entitled "Three-Axis Force Sensor Structure Using Force Sensor and Method of Measuring Force and Moment by Its Structure, hereinafter referred to as Prior Art 1"), at least four force sensor distributions And a bumper shaft connected to an upper portion of the pressing member and configured to transmit an external force to the pressing member. The pressing member and the bumper shaft, 1 force sensor film, a lower plate positioned below the first force sensor film and reacting against the action of the external force, a first hole contacting the edge of the lower plate and projecting the upper end of the bumper shaft to the outside, A second hole for drawing a signal input / output line connected to the first force sensor film is formed on the upper surface of the bumper,

Wherein the upper plate and the lower plate are made of a metal or a polymeric material, wherein the first force sensor film and the upper plate sandwich the first force sensor film and the bumper.

KR 10-1169940 B1

The first problem of the prior art 1 is that when processing user input by a contact with a user who is a person, the user is forced to deliver a cold or hard feeling and is insufficient in terms of sensitivity quality. The first problem is that the components are made of metal, It has a second problem that it is relatively high.

According to an aspect of the present invention, there is provided an input unit including a sheet having a function of restoring an initial shape when a user input is applied to one surface of the sheet, An image pickup unit for collecting marker images as shot images of the plurality of markers varying corresponding to the user input, and a function of determining the characteristics of the user input by processing the marker images (A) a step in which the pushing input is made to the seat; and a step of inputting the pushing input to the seat; (b) obtaining an initial marker image, which is an image of a marker before the pressing input, and a pressing state marker image, which is a marker image under the pressing input, into the processor unit; (c) calculating a vertical pushing displacement in such a way that the processor compares the initial marker image with the pushing state marker image; (d) determining the equivalent elastic modulus corresponding to the position on the sheet on which the pushing input was made by the processor unit; (e) calculating the vertical force by the pushing input by substituting the values of the vertical pushing displacement and the equivalent elastic modulus into predetermined equations, respectively, to calculate a vertical force A method of estimating a direction force is provided.

Further, the sheet of the present invention may be one of the faces constituting the cushion filled with air therein.

In the step (e), the magnitude of the vertical force by the pushing input may be calculated as a product of the vertical pushing displacement and the equivalent elastic modulus.

Further, the determination of the equivalent elastic modulus in the step (d) may be performed by referring to the processor-added equivalent elastic modulus reference data table, and the equivalent elastic modulus reference data table may include a table on the cushion part And the equivalent elastic modulus may be recorded.

Further, the coordinate value indicating the position of the cushion part on which the pushing input is made according to the present invention can be displayed in a two-dimensional coordinate format.

In addition, the two-dimensional coordinate format may be an x-y plane coordinate system or an r-theta polar coordinate system.

A method for estimating a horizontal force by a push / movement input to an interface device of the present invention includes an input unit including a sheet having a function of restoring an initial shape when a user input is applied to one surface, A plurality of markers arranged on a sheet surface in a predetermined manner; an image pickup unit for collecting marker images as picked-up images of the plurality of markers corresponding to the user input; (Push / move input) path under a push input by using an interface device including a processor unit that performs a function of determining a horizontal force on the seat, Performing the push / move input; (ii) acquiring and inputting an initial marker image before the pressing / movement input and the current marker image being a marker image at a current position on the pressing / moving input path, and inputting the current marker image to the processor unit; (iii) calculating a vertical pushing displacement in which the processor compares the initial marker image with the current marker image; (iv) determining an equivalent elastic modulus corresponding to a current position on the push / movement input path by the processor unit; (v) substituting values of the vertical pushing displacement and the equivalent elastic modulus into a predetermined equation to calculate a vertical force at a current position on the pushing / moving input path; (vi) determining a friction coefficient corresponding to a current position on the push / movement input path by the processor unit; (vii) calculating a horizontal force at a current position on the push / movement input path by substituting a value of the vertical force and the coefficient of friction into a predetermined equation, To estimate the horizontal force by the horizontal force.

In the step (vii), the magnitude of the horizontal force at the current position on the push / movement input path may be calculated as the product of the horizontal force and the coefficient of friction.

Further, the determination of the friction coefficient in the step (vi) may be performed by referring to the friction coefficient reference data table, and the friction coefficient reference data table may include a coordinate value indicating a current position on the pressing / And the friction coefficient may be recorded.

In addition, the coordinate value indicating the current position on the push / move input path can be displayed in a two-dimensional coordinate format.

In addition, the coordinate value indicating the current position on the push / move input path may be displayed in a three-dimensional coordinate format further including the vertical pushing displacement in addition to the two-dimensional coordinate format.

In the step (v), the magnitude of the vertical force at the current position on the push / movement input path may be calculated as the product of the vertical pushing displacement and the equivalent elastic modulus.

Further, the determination of the equivalent elastic modulus in the step (iv) may be performed by referring to the processor-added equivalent elastic modulus reference data table, and the equivalent elastic modulus reference data table may include: And the relationship between the coordinate value and the equivalent elasticity coefficient may be recorded.

The present invention also provides a user input device for processing user input using a method for estimating a horizontal force by a push / movement input to the interface device.

The present invention also provides a pressure sensor for sensing and measuring multi-dimensional pressure using a method of estimating a horizontal force by a push / movement input to the interface device.

The present invention also provides a human-machine interface (HMI) interface device that performs a function of processing a user input and controlling the operation of a device by using a method of estimating a horizontal direction force by pressing / moving input to the interface device to provide.

The present invention provides a first effect that quantitative measurement of various user inputs such as clicks, presses, and movements, a second effect that a sensible quality such as touch and texture can be enhanced by using a soft material, A third effect that the image sensor or the like can be realized at a low cost by using the main components, and the fourth effect that the cushion part can be customized in various shapes according to the user's situation.

1 is a block diagram showing a configuration of an embodiment of an interface device of the present invention.
2 is a side sectional view showing a configuration of an embodiment of the interface device of the present invention.
3 is a bottom view showing a configuration of an embodiment of the interface device of the present invention.
4 is an explanatory diagram showing one embodiment of a user input of a push to the interface device of the present invention and a corresponding marker image.
5 is an explanatory diagram showing one embodiment of a user input of push / move and an associated marker image to the input device of the present invention.

While the invention is susceptible to various modifications and alternative forms, specific embodiments thereof are shown by way of example in the drawings and will herein be described in detail. It should be understood, however, that the invention is not intended to be limited to the particular embodiments, but includes all modifications, equivalents, and alternatives falling within the spirit and scope of the invention. Like reference numerals are used for like elements in describing each drawing.

It is to be understood that when an element is referred to as being "connected" or "connected" to another element, it may be directly connected or connected to the other element, . On the other hand, when an element is referred to as being "directly connected" or "directly connected" to another element, it should be understood that there are no other elements in between.

The terminology used in this application is used only to describe a specific embodiment and is not intended to limit the invention. The singular expressions include plural expressions unless the context clearly dictates otherwise.

In the present application, the terms "comprises" or "having" and the like are used to specify that there is a feature, a number, a step, an operation, an element, a component or a combination thereof described in the specification, But do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, or combinations thereof.

The user input in the present invention includes at least one of push, release, and movement.

The user who is the subject of user input in generating user input is not limited to a person but may include a machine, a robot, and other devices. In addition, when the user is a person, it is possible to include not only the hands and feet but also other body parts as the body parts for inputting.

In the case of pressing, the pressing pressure and the holding time are not limited, and the specific values of these parameters may be determined differently depending on the specific configuration of the cushion described later and the application example in which the present invention is used. In the case of depressing, the time required for depressing is not limited to a specific range, but may be determined differently depending on the specific configuration of the cushion described later and the application example in which the present invention is used. Furthermore, when a pressing occurs for a predetermined time, and immediately thereafter, a pressing release occurs, this can be referred to as a click. Movement means moving from a first point to a second point on the surface of the cushion while the user maintains a state of depression, the moving path is not limited to any particular one, and may include straight lines, curves, Circle, ellipse, arc, spline, and the like, but is not limited thereto. However, as will be described later, an embodiment of a pattern of user input such as pressing, depressing, moving, etc., examines the type of change in the size or shape of each of the plurality of markers 30, , And it is evident that there is no intention to exclude from the present invention any pattern that can not be grouped into the above exemplary pattern.

Referring to Figs. 1 to 3, the interface device 1 assumed in the present invention will be described first. The interface device 1 includes an input unit 20 including a sheet having a function of restoring an initial shape when a user input is applied to one side and the user input is canceled, a plurality An image sensing unit 40 for collecting marker images 2 as photographed images of the plurality of markers 30 corresponding to the user input, And a processor unit for performing a function of determining characteristics of the input.

Referring to FIG. 2, the input unit 20 includes a sheet, which may be a stretchable material that receives a user input to cause a local displacement. The stretchability is preferably measurable for each part of the sheet. That is, the degree of stretchability can be represented by an equivalent elastic modulus, which will be described later. It is preferable that the sheet is made of a material that can transmit light to a certain degree or more. However, it is not excluded that the sheet is made of an opaque material. In this case, the illuminating unit should be further provided in the imaging unit 40 Will be described later. In addition, the size of the sheet must be determined in conjunction with the size of the area where the user input occurs.

In an embodiment in which the sheet is installed to constitute the input unit 20, the sheet is made of a soft cloth material, and the edge can be fixed so as to change according to user input. This means that the edge is a fixed end and the central part where actual user input is made is not restrained. The mechanism structure (edge fixing portion) for fixing the edge of the sheet is not limited to a specific one. For example, a through hole for inserting the edge of the sheet is formed in the center of the cylinder, and a tension is applied to the sheet by rotating the cylinder after inserting the edge of the sheet. The shape of the sheet is square, If this is not limited to a square, then these four edge fixtures will be able to apply a total of four equations. In view of the fact that when the magnitude of the tension applied to the seat by the edge fixing portion gradually decreases as the practice of the present invention continues, a problem may arise in the estimated values of the horizontal and vertical forces as described later, The installation of the sheet by means of the roller must be reliably performed to maintain the tension even in the long term use. Further, as shown in Fig. 2, the air cushion may be provided on the substrate.

Further, the sheet of the present invention may be one of the faces constituting the cushion filled with air therein. That is, the input unit 20 is configured so as to have a cushion shape. In this case, the cushion can be formed so as to have a predetermined shape by injecting air into the cushion. This shape need not be limited to a particular shape, as long as it is possible for user input to occur, not to mean user to person. In determining the material of the cushion, it is necessary to consider that, in the case where the surface is too rough or too slippery, the user may be inconvenient in the process of generating the user input, or the user input may not be properly generated. Furthermore, the material should be determined so as to satisfy the sensibility quality such as touch, as well as the convenience of the user. The material of the cushion may be stretchable. The cushion is preferably made of a material which can transmit light to a certain extent or more. However, it is not excluded that the cushion is made of opaque material. In such a case, the illumination section should be further provided in the imaging section 40, Will be described later.

Referring to Figs. 2 and 3, a plurality of markers 30 are provided on one side of the sheet. The plurality of markers 30 should be changed according to the surface of the sheet to be deformed when a user input is generated on the sheet. In the case of the cushion type, it is preferable that the cushion is attached to the upper surface of the cushion, but it is not excluded that the cushion is formed on the other side such as the side surface of the cushion. In the case of a surface cushion which is not exposed to the outside of the seat, it is preferable to be provided on the back surface (inner surface). When the surface cushion is provided on the surface (surface) This can happen.

Each of the plurality of markers 30 may have various shapes such as a circle, an ellipse, and a square. The thickness of the markers 30 need not be limited to a specific range. It is contemplated that the plurality of markers 30 may be installed over an entire area of the sheet or a smaller area thereof so as to recognize various patterns of user input. Needless to say, in the case where the pattern of the user input is limited to a simple pattern such as simple depression / depression, one or two markers may be sufficient. In this case, the total number of the plurality of markers 30 should be determined in consideration of the size of each of the markers and the installation spacing between the markers. If the size of each of the markers becomes small, or the spacing between the markers becomes small A larger number of markers will need to be installed.

The plurality of markers 30 interlock with each other in accordance with the surface shape of the sheet to be deformed to change the position (absolute position) and attitude with respect to the reference position when user input is made to the sheet. The attitude is a concept including an angle with respect to a horizontal plane and an inclination direction. However, even if user input occurs, the shape and size of each of the markers themselves do not change.

The installation pattern of the plurality of markers 30 may have a predetermined number of rows and columns, and may take into account various patterns such as the same spacing between rows and rows, columns and columns, or an arrangement of concentric circles. However, it is obvious that the processing algorithm of the marker image 2 of the processor unit 50 must be changed according to the installation pattern of the plurality of markers 30.

2, the image sensing section 40 performs a function of photographing and collecting a marker image 2 as a photographed image of the plurality of markers 30 corresponding to a user input to the sheet . When a user input is made, the surface shape of the cushion is continuously changed, and the image sensing unit 40 can acquire the marker image 2 which continuously changes with time. Here, the marker image 2 means an image of a plurality of markers as a whole, and the term marker image 2 is used separately from the image of each of the markers. The imaging unit 40 may further perform a function of converting the photographed image into a digital image, and may be an element such as a CCD, but the present invention is not limited to this example. The required resolution is not limited to a specific range, but it is necessary to have a minimum resolution sufficient to identify a change in the marker image 2 as user input occurs. Further, it is possible to further include an illumination unit for providing illumination to the installation area of the plurality of markers 30. In the case where the input device of the present invention is used in a dark environment even when the sheet is a light-transmitting material, A change of the marker image 2 can not be identified unless it is a special case such as the use of a camera or the like. In order to prepare for such a case, the illumination unit may be necessarily provided. The illumination unit may be an LED lamp or the like and may be configured to automatically turn off when a user input does not occur over a predetermined time. In the case of the input unit 20 of the cushion type, the imaging unit 40 may be provided inside the cushion or outside the cushion. In the latter case, however, the imaging unit 40 may be provided between the imaging unit 40 and the plurality of patterns It is clear that there should be more than a certain level of opaque components, or external components should not be displayed.

The processor unit 50 processes the marker image 2 to determine the characteristics of the user input. The function of the processor unit 50 is in line with the function of estimating the vertical force by the pressing input to the interface apparatus 1 of the present invention.

Referring to Fig. 4, a method for estimating the vertical force by pressing input to the interface device 1 of the present invention will be described in detail below.

First, the push input is made to the sheet. The pushing input at this time is preferably in a direction perpendicular to the pressing portion of the sheet - in the direction normal to the sheet surface, but also includes a case in which the pressing input is not perpendicular to the pressing portion of the sheet. Of course, in this case, some additional points should be taken into account in calculating the vertical force on the push input, which will be described later.

Second, the image sensing unit 40 acquires an initial marker image, which is a marker image before the pressing input, and a pressing state marker image, which is a marker image under the pressing input, and inputs the obtained marker image to the processor unit 50. In the initial marker image, the shape and the size of each of the plurality of markers 30 constituting the initial marker image are all the same. However, in the case of the input unit 20 of the cushion type, since the curved surface is formed toward the edge, ), It is noted that the image of each of the markers may be elliptical or the size may be larger or smaller than the average value.

The pressed state marker image will be enlarged in size and changed to an elliptic shape unlike the case where the image of some markers provided around the pressing portion on the sheet on which the pressing input is generated is in the initial state-non-pressing input unactuated state.

Third, the processor unit 50 calculates the vertical pushing displacement in comparison with the initial marker image and the pushing state marker image. As described above, when a pushing input is generated, the image of each marker changes around the pushing input point. In particular, the change in size and shape of the markers close to the pushing input point becomes the greatest. Thus, the vertical pushing displacement can be calculated by comparing the difference between the initial marker image and the pushing state marker image. Normally, when a pressing input is generated, the image of the imaging unit 40 with respect to the marker installed around the relevant region is recognized to be larger. The distance (L0) between the marker and the reference position of the camera in the initial state, the size (D1) of the marker in the pressed state, the marker and the reference position of the camera And the distance L1 between the center and the center. That is, when the push input is generated and the state of L1 < L0 becomes D1 > D0, it is possible to establish the relation of L alpha 1 / D through a predetermined configuration, (K) is determined and L = k / D is established, the vertical pushing displacement generated by the corresponding pushing input can be estimated by determining only D1 after image processing. Of course, the present invention is not limited to this example, and various algorithms for calculating (estimating) the vertical pushing displacement can be considered.

Fourth, the processor unit 50 determines the equivalent elastic modulus corresponding to the position on the sheet on which the pushing input is made.

Since the sheet is preferably made of a stretchable material, when a force is applied to a predetermined portion of the sheet, the sheet is stretched in the direction of the applied force, and the stretchability of the sheet causes a restoring force in the direction opposite to the stretched direction . Further, when the input unit 20 is configured as an air-cushion type, the input unit 20 generates a force that repulss the pushing pressure and restores the original state due to the pressure of the injected air. However, in these embodiments, drawing the free body diagram (FBD) for the input unit 20 can be described as an elastic model that follows the Hook rule as an equivalent model. The law of Hook's law is that, in the case of an elastic body, the magnitude of the restoring force generated when the strain is applied is proportional to the displacement of the applied strain, and the proportional constant (elastic modulus) at that time is determined by the type of the elastic body.

When the input unit 20 is formed by fixing the seat with the edge fixing unit, a portion fixed to the edge fixing unit fixed to the seat may be a displacement of 0, and a portion to which the pressing input is applied may occur And the displacement of the remaining part is different according to the distance from the part where the pushing input is applied. In the present invention, however, it is desired to estimate the magnitude of a locally acting force due to a pressing input applied to a local region of the sheet rather than a total force or an average force applied to the entire sheet. Therefore, It has an indirect effect on the determination of the displacement in the region to be formed, but is meaningless in the present invention. In addition, when pressing input of the same size is applied to various portions of the sheet, the magnitudes of the displacements occurring in the respective portions are different from each other, and this phenomenon means that the equivalent elastic modulus is different for each portion of the sheet. Herein, the equivalent elastic modulus refers to a force applied to a specific portion of the seat, which causes a displacement of the seat, so that the seat can be considered to be equivalent to a spring. At this time, , Which is the equivalent elastic modulus for the site.

In the construction of the air cushion type input section 20, there is no edge portion, and once a pressing input is generated at a specific portion, displacement occurs in all the portions of the sheet constituting the air cushion. However, in the air cushion type embodiment as well as the embodiment having the edge fixing portion, only the magnitude of the force generated on the portion to which the pressing input is applied and the portion to which the pressing input is applied has a meaning . In addition, when the push input of the same size is applied to various portions of the air cushion, the magnitudes of the displacements generated for the respective portions are different from each other. This phenomenon means that the equivalent elastic modulus is different for each portion of the air cushion do. Here, the equivalent elastic modulus means that the air cushion generates a displacement with respect to a force applied to a specific portion, and therefore, the air cushion can be equivalently regarded as a spring spring. At this time, , Which is the equivalent elastic modulus for the site, as described above.

In other words, the equivalent elastic modulus corresponding to the pressing input of the air cushion type or the type having the edge fixing portion varies depending on the position on the sheet or air cushion surface where the pressing input is generated, and furthermore, The coefficient and the position on the sheet or on the air cushion surface will have a predetermined correlation or function relationship.

Therefore, by applying a force of a predetermined magnitude to a specific region on the surface of the sheet or air cushion and directly measuring the displacement, the equivalent elastic modulus can be obtained for the region, It is possible to create an equivalent elastic modulus reference data table expressing the characteristics of the input unit 20 of the corresponding configuration. It is clear that the equivalent elastic modulus reference data table 52 will be made differently depending on the material, shape, and the like of the sheet constituting the input unit 20. [

1, it may be necessary to design the experiment such that, in determining the equivalent elastic modulus, the pressing input is generated in a direction perpendicular to the surface of the sheet or air cushion (normal vector direction). The thus determined elastic modulus reference data table 52 can be said to be more accurate when the actual pressing input actually acts in the vertical direction (normal vector direction) with respect to the surface of the seat or air cushion. Therefore, when the actually generated pushing input does not act in a direction perpendicular to the surface of the seat or air cushion, the estimated value of the vertical force may not reflect the actual value. In order to compensate for this, it is possible to further comprise a predetermined method of correction algorithm in estimating the vertical force in relation to the component in the direction parallel to the surface of the seat or the air cushion by decomposing the pushing input actually generated.

The expression of the equivalent elastic modulus reference data table 52 may be varied. For example, the relationship between the coordinate value indicating the position on the cushion portion on which the pushing input is made and the equivalent elastic modulus may be recorded.

Further, the coordinate value indicating the position on the sheet on which the pushing input is made according to the present invention can be displayed in a two-dimensional coordinate format. Particularly, in the case of the air cushion type embodiment, the middle portion of the air cushion may have a more convex shape than the edge of the air cushion. In this case, it may be considered that the upper surface of the air cushion is three- When the curvature is not large in the convex shape of the center portion, it is appropriate to display it two-dimensionally assuming that there is no difference in the height direction. The two-dimensional coordinate format may be an xy plane coordinate system or an r-theta polar coordinate system, and may be displayed reflecting the actual distance measured from the reference position of the seat, or the serial number of the plurality of markers 30 may be N * It is of course possible that each marker is represented by an ordered pair (n is a natural number equal to or less than N, and m is a natural number equal to or less than M), such as (n, m).

The equivalent elastic modulus reference data table 52 is recorded and stored in a predetermined method and the equivalent elastic modulus is determined by referring to the equivalent elastic modulus reference data table 52 of the processor unit 50 Can be adopted. That is, the position at which the pushing input is generated is calculated and represented as a coordinate value, and the equivalent elasticity coefficient corresponding to the coordinate value can be retrieved and utilized by referring to the equivalent elasticity coefficient reference data table 52.

Fifth, the values of the vertical pushing displacements calculated in the third step and the values of the equivalent elastic moduli determined in the fourth step are substituted into a predetermined equation to calculate (estimate) the vertical force by the pushing input. At this time, the predetermined equation can calculate the magnitude of the vertical force by the pushing input as a product of the vertical pushing displacement and the equivalent elasticity modulus based on a hook rule. However, in order to obtain more precise results, the function relation between the magnitude of the vertical force and the push-in input is not a linear function relation according to the Hook's law but a quadratic function relation or other linear or nonlinear function relation It is not excluded.

Hereinafter, a method for estimating the horizontal force at the current position on the movement input (push / movement input) path under the push input using the interface device 1 of the present invention will be described in detail. When the push / move input acts on the seat or the air cushion surface, the seat or air cushion is deformed in the vertical direction by the push / move input, and the restoring force is received. Is as described above, and furthermore, the horizontal component of the push / movement input provides a cause to move while overcoming the frictional force of the surface of the seat or air cushion. Therefore, the present invention makes use of the principle that such a frictional force can be estimated by considering it as a horizontal force.

The detailed configuration of the interface device 1 to be applied is not different from that applied to the aforementioned method of estimating the vertical force.

5, each step in the method for estimating the horizontal force by the push / movement input to the interface apparatus 1 of the present invention will be described in detail below.

First, the push / move input is performed on the sheet. In this respect, there is a difference from the user input which is problematic in the estimation method of the vertical direction force described above. The user input in the horizontal direction force estimation method of the present invention is limited to the movement input under the push input. That is, when there is only a pressing input and there is no movement, considering that the friction coefficient described later is not the static friction coefficient but the moving friction coefficient, even if the horizontal force is calculated, the value is not meaningful. Therefore, it may be desirable to discard the horizontal force that is automatically calculated by the processor unit 50 in the case of a user input with only a push input, without using it. In addition, user input such as sliding in fine contact with the sheet or air cushion surface without pressing input is meaningless in the present invention. However, it is not necessary that the magnitude of the pushing input continues to be a constant value during the movement input. In the horizontal force estimation method of the present invention, the vertical pushing displacement due to the pushing input is estimated in real time, , It is used for estimating the vertical direction force, and the estimated vertical force is used for estimating the horizontal direction force, so that the magnitude of the push input may be changed.

Second, the imaging unit 40 acquires an initial marker image before the pressing / movement input and a current marker image that is a marker image at the current position on the pressing / moving input path, and inputs the acquired marker image to the processor unit 50. In the case of the air-cushion type input unit 20, since the curved surface is formed as the edge of the initial marker image is formed, the number of the markers 30 Note that for a portion located at the edge, the image of each of the markers may be elliptical or the size may be larger or smaller than the average value.

The current marker image means the marker image at the present time in which the push input and the movement input act simultaneously (this input is referred to as push / move input). The meaning of 'present time' means that, in the case of movement input, the actual acting position of the force continuously changes continuously in real time. Once the current marker image is used to estimate the magnitude of the vertical force or horizontal force, the current marker image will be the marker image of the past point of view. This current marker image may estimate the magnitude of the vertical force or the horizontal force and then discard or record it, but since the data actually needed and useful to the user will be the magnitude of the vertical force or the horizontal force, It would be desirable to discard the current marker image. Unlike the case where the image of some markers installed in the vicinity of the pressing / moving input action area on the sheet or cushion (air) is not in the initial state-pushing / moving input non-operating state, its size becomes larger and its shape is changed to an ellipse will be.

Third, the processor unit 50 calculates a vertical pushing displacement in comparison with the initial marker image and the current marker image. As described above, when the push / move input is generated, the image of each marker changes in the center of the region where the push / move input is performed. In particular, the change in the size and shape of the markers close to the push / move input point becomes greatest. Thus, the vertical pushing displacement can be calculated by comparing the difference between the initial marker image and the current marker image under the push / move state. Normally, when the push / move input occurs, the image of the image pickup section 40 with respect to the markers installed around the relevant region is recognized to be larger. The distance (L0) between the marker and the reference position of the camera in the initial state, the size (D1) of the marker in the current state of the push / And the distance (L1) from the camera reference position. That is, when the push / move input is generated and the state of L1 < L0 becomes D1 > D0, it is possible to establish the relationship of L? 1 / D through a predetermined configuration. If the relational expression of L = k / D is established by determining the proportionality constant k of 1 / D, then if only D1 is determined through image processing, the vertical pushing displacement caused by the corresponding push / It is possible. Of course, the present invention is not limited to this example, and various algorithms for calculating (estimating) the vertical pushing displacement can be considered.

Fourth, the processor unit 50 determines the equivalent elastic modulus corresponding to the current position on the push / movement input path.

Since the sheet is preferably made of a stretchable material, when a force is applied to a predetermined portion of the sheet, the sheet is stretched in the direction of the applied force, and the stretchability of the sheet causes a restoring force in the direction opposite to the stretched direction . Further, when the input unit 20 is configured as an air-cushion type, the input unit 20 generates a force to repel the push / move input and restore the original state due to the pressure of the injected air. However, in these embodiments, drawing the free body diagram (FBD) for the input unit 20 can be described as an elastic model that follows the Hook rule as an equivalent model. The law of Hook's law is that, in the case of an elastic body, the magnitude of the restoring force generated when the strain is applied is proportional to the displacement of the applied strain, and the proportional constant (elastic modulus) at that time is determined by the type of the elastic body.

In the case where the input unit 20 is constructed by fixing the seat with the edge fixing unit, a portion fixed to the edge fixing unit fixed to the seat may be a displacement of zero, And the displacement of the remaining part is different according to the distance from the part where the push / movement input is applied. However, since the present invention estimates the magnitude of a locally acting force due to a push / movement input applied to a local portion of the seat rather than a total force or an average force applied to the entire seat, Although this indirectly affects the determination of the displacement in the region to which the movement input is applied, the meaning is small in the present invention. In addition, when the push / movement input of the same size is applied to various portions of the sheet, the magnitudes of the displacements generated for the respective portions are different from each other. This phenomenon means that the equivalent elastic modulus is different for each portion of the sheet do. Herein, the equivalent elastic modulus refers to a force applied to a specific portion of the seat, which causes a displacement of the seat, so that the seat can be considered to be equivalent to a spring. At this time, , Which is the equivalent elastic modulus for the site.

In the configuration of the air cushion type input section 20, there is no edge portion, and once a pressing / moving input is generated at a specific portion, displacement occurs in all the portions of the sheet constituting the air cushion. However, in the air cushion type embodiment as well as the embodiment having the edge fixing portion, the displacement generated at the portion to which the pressing / movement input is applied and the magnitude of the force acting on the portion to which the pressing / It has this meaning. In addition, when push / movement inputs of the same size are applied to various portions of the air cushion, the magnitudes of displacements occurring at the respective portions are different from each other. This phenomenon is considered to be caused by the fact that the equivalent elastic modulus . Here, the equivalent elastic modulus means that the air cushion generates a displacement with respect to a force applied to a specific portion, and therefore, the air cushion can be equivalently regarded as a spring spring. At this time, , Which is the equivalent elastic modulus for the site, as described above.

In other words, the equivalent elastic modulus of the air cushion type or the type having the edge fixing portion corresponding to the pressing / movement input portion varies depending on the position on the sheet or air cushion surface where the pushing / moving input is generated, , The equivalent elastic modulus and the position on the sheet or air cushion surface have a predetermined correlation or function relationship.

Therefore, by applying a force of a predetermined magnitude to a specific region on the surface of the sheet or air cushion and directly measuring the displacement, the equivalent elastic modulus can be obtained for the region, It is possible to create the equivalent elastic modulus reference data table 52 expressing the characteristics of the input unit 20 of the corresponding configuration. It is clear that the equivalent elastic modulus reference data table 52 will be made differently depending on the material, shape, and the like of the sheet constituting the input unit 20. [

Here, in the creation of the equivalent elastic modulus reference data table 52, it is preferable to perform the experiment using the user input made only of the above-described push input. In the case of the push / move input, since it always has a component in a direction parallel to the surface of the seat or air cushion, the equivalent elastic modulus is the displacement in the direction perpendicular to the sheet or air cushion surface (Normal vector direction) with respect to the surface of the seat or air cushion, it is reasonable to design the experiment to occur.

Thus, since the actually generated push / move input does not generally act in the vertical direction with respect to the surface of the seat or air cushion, the estimated value of the vertical force may not reflect actuality. In order to compensate for this, it is necessary to decompose the push / move input actually generated in a vector manner so that the influence of the component in the direction parallel to the surface of the seat or air cushion is minimized or eliminated. A correction algorithm may be further provided.

The expression of the equivalent elastic modulus reference data table 52 may be various. For example, the relationship between the coordinate value indicating the position on the cushion portion on which the push / movement input is made and the equivalent elastic modulus may be recorded have.

Further, the coordinate value indicating the position on the sheet on which the push / move input of the present invention is made can be displayed in a two-dimensional coordinate format. Particularly, in the case of the air cushion type embodiment, the middle portion of the air cushion may have a more convex shape than the edge of the air cushion. In this case, it may be considered that the upper surface of the air cushion is three- When the curvature is not large in the convex shape of the center portion, it is appropriate to display it two-dimensionally assuming that there is no difference in the height direction. The two-dimensional coordinate format may be an xy plane coordinate system or an r-theta polar coordinate system, and may be displayed reflecting the actual distance measured from the reference position of the seat, or the serial number of the plurality of markers 30 may be N * It is of course possible that each marker is represented by an ordered pair (n is a natural number equal to or less than N, and m is a natural number equal to or less than M), such as (n, m).

Fifth, the values of the vertical pushing displacement and the equivalent elastic modulus are substituted into a predetermined equation to calculate the vertical force at the current position on the pushing / moving input path. At this time, the predetermined equation can calculate the magnitude of the vertical force by the pushing input as a product of the vertical pushing displacement and the equivalent elasticity modulus based on a hook rule. However, in order to obtain more precise results, the function relation between the magnitude of the vertical force and the push-in input is not a linear function relation according to the Hook's law but a quadratic function relation or other linear or nonlinear function relation It is not excluded.

Sixth, the processor unit 50 determines a friction coefficient corresponding to the current position on the push / movement input path.

At this time, the determination of the friction coefficient is made by the processor unit 50 with reference to the friction coefficient reference data table, and the friction coefficient reference data table 53 has a coordinate value And the friction coefficient may be recorded. Specifically, the determination of the coefficient of friction is performed such that the processor unit 50 receives the coordinate value indicating the current position on the pressing / moving input path, and then refers to the coefficient of friction reference data table 53 to calculate the coordinate value And selects the corresponding coefficient of friction.

In the horizontal direction force estimation method of the present invention, since the user input that is a problem is a push / shift input, the friction coefficient is not a static friction coefficient but a dynamic friction coefficient. The coefficient of friction is different depending on the position on the sheet or the air cushion surface. The reason why such a result is generated even when each part of the seat or the air cushion is made of a single material is that the frictional force generated when the pressing / The fact that the seat 21 or the air cushion is limited to a predetermined shape in constituting the input section 20 of the present invention influences the generation of such a displacement.

That is, even if the same type of push / move input occurs, the frictional force must be estimated differently depending on the position on the sheet 21 or air cushion surface, so that the actually measured frictional coefficient is also on the sheet 21 or on the air cushion surface It is different from place to place. Creation of the friction coefficient reference data table 53 can be performed through designed experiments. If the friction coefficient is determined only depending on the position of the seat 21 or the air cushion surface, the pressing / moving input to the seat 21 or the air cushion surface is applied to the seat 21 or to each part on the surface of the air cushion (The use of a force sensor or the like), and the ratio of the horizontal component to the vertical component is regarded as a coefficient of friction. Of course, in the case of the vertical component, it is of course possible to use the estimated value determined using the vertical force estimation method of the present invention as described above.

The coordinate value indicating the current position on the push / move input path can be displayed in a two-dimensional coordinate format. The two-dimensional coordinate format may be an x-y planar coordinate system or an r-theta polar coordinate system, but does not exclude the application of other display formats.

In addition, the coordinate value indicating the current position on the push / move input path may be displayed in a three-dimensional coordinate format further including the vertical pushing displacement in addition to the two-dimensional coordinate format. This is because, as the vertical component force magnitude increases with respect to the push / move input, the contact surface deforms to a v-shape. Such deformation of the seat 21 or the air cushion surface portion causes a movement / , The vertical pushing displacement can be additionally set as a variable for determining the friction coefficient. In this case, the coefficient of friction will be a tridimensional function.

Seventh, the horizontal force at the current position on the push / movement input path is calculated by substituting the values of the friction force coefficients determined in the vertical force and the sixth step into a predetermined equation. At this time, the magnitude of the horizontal force at the current position on the push / movement input path can be calculated as the product of the horizontal force and the coefficient of friction.

The processor unit 50 quantitatively determines the user input as described above and can utilize the result in various ways. In particular, the processor unit 50 generates the output signal 3 containing the quantitative information about the user input, 4). At this time, the output signal 3 provided to the external device 4 or the like may be an electric signal, but is not limited thereto.

The user input device for processing user input using the method of estimating the horizontal force by the push / move input to the interface device 1 of the present invention can basically have a function of estimating the vertical force The horizontal direction force estimated by such a configuration can be utilized in the implementation of a pointer for moving the cursor on the screen and the vertical direction force can be utilized in such a manner as to execute a specific function corresponding to the size thereof , It is not limited to these examples.

The pressure sensor for sensing and measuring the multidimensional pressure using the method of estimating the horizontal force by the pushing / moving input to the interface device 1 of the present invention can basically have the function of estimating the vertical force , The horizontal force and the vertical force estimated by this configuration can quantify the magnitude of the applied input. This makes it possible to replace pressure sensors and force sensors with complex configurations.

In a human-machine interface (HMI) interface device that performs user input processing and controls device operation using a method of estimating a horizontal force by pressing / moving input to the interface device 1 of the present invention, The device to be controlled can be set to various types such as a robot, a wheelchair, and a display device. For example, when the human-machine interface device of the present invention is implemented in a wheelchair, when the vertical force is smaller than a predetermined value, the sensed horizontal force can be utilized for wheel drive control for movement, If the force is greater than the predetermined value, the sensed horizontal force may be utilized to actuate the hydraulic module and operate the link up the stairs, but this is not limiting.

The foregoing description sets forth the best mode of the invention, and is provided to illustrate the invention and to enable those skilled in the art to make and use the invention. The written description is not intended to limit the invention to the specific terminology presented.

Thus, while the present invention has been described in detail with reference to the above examples, those skilled in the art will be able to make adaptations, modifications, and variations on these examples without departing from the scope of the present invention. In other words, in order to attain the intended effect of the present invention, all the functions shown in the drawings are separately included or all the steps shown in the drawings are not necessarily followed in the order shown, and the technical scope of the present invention In the following description.

1: Interface device
2: Marker image
3: Output signal
4: External device
8: Press the user input
9: User input of push / move
20:
21: Sheet
22: substrate
30: Marker
40:
50:
51: Processor
52: Equivalent elastic modulus standard data table
53: Friction coefficient reference data table

Claims

An input unit including a sheet having a function of receiving user input on one side and restoring the original shape when the user input is dissolved;
A plurality of markers arranged in a predetermined manner on one surface of the sheet,
An image pickup unit for collecting marker images as shot images of the plurality of markers varying corresponding to the user input,
A processor unit for processing the marker image to determine a characteristic of the user input,
A method for estimating a vertical force by a push input using an interface device comprising:
(a) the push input is performed on the sheet;
(b) obtaining an initial marker image, which is a marker image before the pressing input, and a pressing state marker image, which is a marker image under the pressing input, into the processor unit;
(c) calculating a vertical pushing displacement in such a way that the processor compares the initial marker image with the pushing state marker image;
(d) determining the equivalent elastic modulus corresponding to the position on the sheet on which the pushing input was made by the processor unit;
(e) substituting values of the vertical pushing displacement and the equivalent elastic modulus into a predetermined equation, respectively, to calculate the vertical force by the pushing input;
And a controller for determining a vertical force based on a pressing input to the interface device.

Wherein the seat is one of the surfaces constituting a cushion filled with air inside. 2. A method for estimating a vertical force by a pressing input to an interface device, comprising:

The method according to claim 1,
Wherein in the step (e), a magnitude of the vertical force by the pushing input is calculated as a product of the vertical pushing displacement and the equivalent elastic modulus. How to estimate.

The method according to claim 1,
The determination of the equivalent elastic modulus in the step (d) may be made by referring to the processor-added equivalent elastic modulus reference data table,
Wherein the equivalent elastic modulus reference data table records a relationship between a coordinate value indicating a position on the sheet on which the pushing input is made and the equivalent elastic modulus, How to estimate.

The method of claim 4,
Wherein a coordinate value indicating a position on the sheet on which the pushing input is made is displayed in a two-dimensional coordinate format.

The method of claim 5,
Wherein the two-dimensional coordinate format is an xy plane coordinate system or an r-? Polar coordinate system.

An input unit including a sheet having a function of receiving user input on one side and restoring the original shape when the user input is dissolved;
A plurality of markers arranged in a predetermined manner on one surface of the sheet,
An image pickup unit for collecting marker images as shot images of the plurality of markers varying corresponding to the user input,
A processor unit for processing the marker image to determine a characteristic of the user input,
A method for estimating a horizontal force at a current position on a movement input (push / movement input) path under a push input using an interface device comprising:
(i) performing the push / move input on the sheet;
(ii) acquiring and inputting an initial marker image, which is a marker image before the pressing / movement input, and a current marker image, which is a marker image at a current position on the pressing / moving input path, to the processor unit;
(iii) calculating a vertical pushing displacement in which the processor compares the initial marker image with the current marker image;
(iv) determining an equivalent elastic modulus corresponding to a current position on the push / movement input path by the processor unit;
(v) substituting values of the vertical pushing displacement and the equivalent elastic modulus into a predetermined equation to calculate a vertical force at a current position on the pushing / moving input path;
(vi) determining a friction coefficient corresponding to a current position on the push / movement input path by the processor unit;
(vii) calculating a horizontal force at a current position on the push / movement input path by substituting the values of the vertical force and the friction coefficient into a predetermined equation;
And an input device for inputting the horizontal force to the interface device.

The method of claim 7,
Wherein in the step (vii), the magnitude of the horizontal force at the current position on the push / movement input path is calculated as the product of the horizontal force and the coefficient of friction. Of the horizontal force.

The method of claim 7,
The determination of the friction coefficient in the step (vi) may be made by referring to the friction coefficient reference data table,
Wherein the friction coefficient reference data table records a relationship between a coordinate value indicating a current position on the pushing / moving input path and the friction coefficient, wherein the horizontal force due to the pressing / How to estimate.

The method of claim 9,
And a coordinate value indicating a current position on the push / move input path is displayed in a two-dimensional coordinate format. A method for estimating a horizontal force by a push / move input to an interface device.

The method of claim 10,
Wherein the two-dimensional coordinate format is an xy plane coordinate system or an r-? Polar coordinate system.

The method of claim 10,
Wherein the coordinate value indicating the current position on the push / move input path is displayed in a three-dimensional coordinate format further including the vertical direction pushing displacement in addition to the two-dimensional coordinate form. A method for estimating a horizontal force by a movement input.

The method of claim 7,
Wherein the step (v) calculates the magnitude of the vertical force at the current position on the push / movement input path as a product of the vertical pushing displacement and the equivalent elastic modulus, A method for estimating a horizontal force by an input.

The method of claim 7,
The determination of the equivalent elastic modulus in the step (iv) may be performed by referring to the processor-added equivalent elastic modulus reference data table,
Wherein the equivalent elastic modulus reference data table records a relationship between a coordinate value indicating a current position on the push / movement input path and the equivalent elastic modulus. How to estimate force.

15. The method of claim 14,
And a coordinate value indicating a current position on the push / move input path is displayed in a two-dimensional coordinate format. A method for estimating a horizontal force by a push / move input to an interface device.

16. The method of claim 15,
Wherein the two-dimensional coordinate format is an xy plane coordinate system or an r-? Polar coordinate system.

A user input device for processing user input using a method for estimating a horizontal force by a push / move input to an interface device of claim 7.

A pressure sensor for sensing and measuring multidimensional pressure using a method for estimating a horizontal force by a push / move input to the interface device of claim 7.

A human machine interface (HMI) interface device for processing user input and controlling device operation using a method for estimating a horizontal force by a push / move input to an interface device of claim 7.