WO2018207490A1

WO2018207490A1 - Contactless three-dimensional touch panel, contactless three-dimensional touch panel system, contactless three-dimensional touch panel control method, program and recording medium

Info

Publication number: WO2018207490A1
Application number: PCT/JP2018/012792
Authority: WO
Inventors: 西田　誠
Original assignee: 株式会社ネットアプリ
Priority date: 2017-05-11
Filing date: 2018-03-28
Publication date: 2018-11-15
Also published as: JP2018190306A; JP6233941B1

Abstract

Provided are: a touch panel which has a simple device configuration, can be produced at a low cost, and is unlikely to fail; and a touch panel system and the like which use said touch panel. This contactless three-dimensional touch panel 1 is equipped with: a movement space 10 which is the range through which an object A to be measured moves; a colored first band 20 which is positioned in the YZ plane in the movement space; a colored second band 30 positioned in the ZX plane; a first reflecting mirror 70 positioned in a location that faces the first band; a second reflecting mirror 71 positioned in a location that faces the second band; a third imaging means 80 for simultaneously capturing both an image 81 of the first band reflected by the first reflecting mirror and an image 82 of the second band reflected by the second reflecting mirror; and an image analysis means 60 for calculating the position of the object to be measured in the YZ coordinate plane and the position thereof in the ZX coordinate plane, by analyzing the images captured by the third imaging means.

Description

Non-contact type three-dimensional touch panel, non-contact type three-dimensional touch panel system, non-contact type three-dimensional touch panel control method, program, and recording medium

The present invention provides a non-contact type three-dimensional touch panel, a non-contact type three-dimensional touch panel system, and a non-contact type three-dimensional touch panel that have a simple apparatus configuration, low manufacturing cost, are unlikely to fail, and can be used outdoors. The present invention relates to a control method, a program, and a recording medium.

Electronic information devices such as computers and portable information terminals may be equipped with a touch panel that combines an input device and a display device (display).
There are various types of touch panels such as a capacitive method, a resistive film method, an ultrasonic surface acoustic wave method, and an optical method (infrared optical imaging method).
For example, in the capacitance type (see Patent Document 1), a matrix-like electrode pattern is arranged in the panel, and a coordinate is detected based on a change in capacitance generated when the user touches with a finger or the like. It has become.
In the resistance film type (see Patent Document 2), a voltage is applied to both ends of two opposed films, and coordinates are determined based on a change in voltage at a position where the films touch each other when the user touches. It is a mechanism to detect.
In addition, in the optical method (see Patent Document 3), infrared LEDs and image sensors (cameras) are arranged on the left and right edges of the top side of the panel, respectively, and the retroreflected light reflects incident light in the direction of incidence on the left, right, and bottom sides of the panel. Place the tape. The image sensor detects the shadow that occurs when the user touches the panel with a finger or the like to shield infrared light, and coordinates are detected by triangulation.
These touch panels are for detecting coordinates on the XY plane, and can be called a two-dimensional touch panel.

Furthermore, development of a three-dimensional touch panel that can be operated in a three-dimensional space (XYZ space) is progressing by applying the technology of the two-dimensional touch panel.
For example, Patent Document 4 discloses a proximity / contact sensor having a two-layer structure composed of a first detection means and a second detection means. This sensor measures the electrical change (for example, change in capacitance, etc.), identifies the presence or absence and the position of the object by the first detection means, and detects the object by the second detection means. The contact position and the pressing force are specified. A single sensor can accurately detect a series of movements from approaching, touching, and pressing, and also enables pointing using a target object for images with a three-dimensional effect or depth, such as a three-dimensional image. .

Japanese Patent No. 5306059 Japanese Unexamined Patent Publication No. 2014-134454 Japanese Patent No. 5406990 International Publication No. 2014/080924

However, each of the above conventional techniques has a complicated apparatus configuration, and thus has a problem that the manufacturing cost increases and a problem that a failure easily occurs. In particular, in consideration of outdoor use, it is necessary to have high waterproofness. However, the above-described conventional technology is easily affected by moisture, and there is a problem that operation accuracy is lowered.
In addition, since all of the above conventional techniques calculate the two-dimensional or three-dimensional coordinates of the position of the finger or the like when the user's finger or the like touches the electrode or panel, it is necessary to pay attention to the hygiene aspects of restaurants and the like There is also a problem that it is not necessarily suitable for use in a certain place.

In view of the above problems, the present invention is a non-contact type three-dimensional touch panel, a non-contact type three-dimensional touch panel system, a non-contact type, which has a simple device configuration, low manufacturing cost, is unlikely to fail, and can be used outdoors. It is an object to provide a control method, a program, and a recording medium for a three-dimensional touch panel.

The non-contact type three-dimensional touch panel of the present invention is an object to be measured in an XYZ three-dimensional coordinate system in which the X-axis direction is the left-right direction, the Y-axis direction is the up-down direction, and the Z-axis direction is the front-back direction. Is a moving space that is a moving range, a colored first strip disposed in the YZ plane along the traveling space, and a colored second strip disposed in the ZX plane along the traveling space. A first reflecting mirror disposed at a position facing the first belt member across the moving space, and a second reflecting mirror disposed at a position facing the second belt member across the moving space And third imaging means for simultaneously photographing both the image of the first band reflected by the first reflecting mirror and the image of the second band reflected by the second reflecting mirror, and the third Image analysis means for calculating the position on the YZ coordinate and the position on the ZX coordinate of the object to be measured by analyzing the image captured by the imaging means The features.

Further, the first belt body and / or the second belt body is divided into two or more in the Z-axis direction.
Further, the position of the first belt body and / or the second belt body in the Z-axis direction is variable.
Further, the first belt body and / or the second belt body is divided into two or more in the longitudinal direction, and the position of each divided portion in the Z-axis direction is variable.
Further, the color of the first band and / or the second band is a dark color.
The first belt and / or the second belt may be an image electrically displayed on a video display device.
Moreover, the said imaging | photography means is accommodated in the inside of the box provided with a slit, and it image | photographs through the said slit, It is characterized by the above-mentioned.

Moreover, the cushioning material which has translucency and a softness | flexibility is provided, The said cushioning material is arrange | positioned in the said movement space, It is characterized by the above-mentioned.
Further, when the video analysis unit analyzes the video shot by the third shooting unit, the video is divided into a plurality of pixels, and pixels having a color different from the color of the band are X, Y, Z If a certain number or more are not continuous in any of the axial directions, these pixels are determined to be noise and position coordinates are not calculated.
In addition, when the video analysis unit analyzes the video captured by the third imaging unit, the video analysis unit divides the video into a plurality of pixels, and is based on the absolute value of the difference between the RGB values of two adjacent pixels. Then, the position on the YZ coordinate and the position on the ZX coordinate of the object to be measured are calculated.

A non-contact type three-dimensional touch panel system according to the present invention includes the non-contact type three-dimensional touch panel, a video reproduction device arranged at a position where a user can visually recognize a video display surface through the moving space, and the video analysis unit. Based on the calculated position on the YZ coordinate of the object to be measured and the position on the ZX coordinate, the position where the user moved the object to be measured corresponds to the position on the video display surface, Video processing means for changing the video displayed on the video display surface.
The video is a pointer.
In addition, two or more video reproduction apparatuses are provided.
The video display surface may be a curved surface, and the first belt body and / or the second belt body may be curved corresponding to the curvature of the curved surface.
In addition, a translucent plate is provided between the moving space and the video reproduction device.

The non-contact type three-dimensional touch panel control method of the present invention is a coordinate measurement target in an XYZ three-dimensional coordinate system in which the X-axis direction is the left-right direction, the Y-axis direction is the up-down direction, and the Z-axis direction is the front-back direction. A colored first belt in the YZ plane and a colored second belt in the ZX plane are arranged along the movement space, which is the range in which the object to be measured moves. The first reflecting mirror is disposed at a position facing the first belt body, the second reflecting mirror is disposed at a position facing the second belt body, and the third photographing means is reflected by the first reflecting mirror. Simultaneously capturing both the image of the first band and the image of the second band reflected by the second reflecting mirror, and the image analysis means analyzes the image captured by the third imaging means. And calculating the position on the YZ coordinate and the position on the ZX coordinate of the object to be measured.
The program of the present invention causes the computer connected to the non-contact type three-dimensional touch panel to execute the control method of the non-contact type three-dimensional touch panel.
The recording medium of the present invention is characterized in that the program is recorded.

The non-contact type three-dimensional touch panel and the three-dimensional touch panel system of the present invention have a first belt body and a second belt body arranged along a movement space, and photograph an object to be measured moving in the movement space with an imaging means. To detect the YZ and ZX coordinates of the object to be measured. Since the apparatus configuration is simple, the manufacturing cost is low, failure is unlikely to occur, and an increase in size is easy. Also, unlike various methods such as the conventional capacitance method, resistive film method, ultrasonic surface acoustic wave method, and optical method (infrared optical imaging method), it is difficult to be affected by the surrounding environment, so it is used outdoors. Even if it works correctly. Moreover, since it operates without contact, it is hygienic.
The first belt and / or the second belt is divided into two or more in the Z-axis direction, the position in the Z-axis direction is variable, and the first belt and / or the second belt is 2 in the longitudinal direction. If the position in the Z-axis direction of each divided part is made variable, the sensitivity of the touch panel can be adjusted freely.
In addition, if the color of the first belt and / or the second belt is dark, it is possible to prevent malfunction caused by the shadow of the object to be measured being reflected on the first belt and / or the second belt. .
In addition, if the first band and / or the second band is an image electrically displayed on the video display device, the shape of the band can be freely adjusted.

Further, if the photographing means is accommodated in the box provided with the slit and the photographing is performed through the slit, it can be operated accurately without being influenced by the surrounding illumination environment.
Further, by arranging a cushioning material having translucency and flexibility in the moving space, the user can operate the three-dimensional touch panel while having a sense of touching the image on the image display surface.
Further, when the video analysis unit analyzes the video shot by the shooting unit, the video is divided into a plurality of pixels, and the pixel having a color different from the color of the band is in any of the X, Y, and Z axis directions. If a certain number of pixels are not consecutive in that direction, those pixels are judged as noise and position coordinates are not calculated, or the absolute value of the difference between the RGB values of two adjacent pixels If the position on the YZ coordinate and the position on the ZX coordinate of the measurement object are calculated based on the size, the analysis processing speed can be improved.
If the video displayed on the video playback device is a pointer, the pointer moves on the screen of the video playback device according to the position of the object to be measured, and visual operations such as clicking various buttons on the screen are possible. become.
If two or more video playback devices are provided, different videos can be projected on each video playback device, or the video of each video playback device can be combined into one large video, so the video variations Can be increased.
If the video display surface is formed of a curved surface and the first band and / or the second band are curved, it is possible to cope with a so-called flexible display.
If a translucent plate (for example, window glass) is provided between the moving space and the video reproduction device, the moving space and the video reproduction device can be arranged in different spaces partitioned by the plate. it can.

Side view (a) showing the configuration of the first embodiment of the non-contact type three-dimensional touch panel and the three-dimensional touch panel system, the image (b) of the first belt taken by the first photographing means, and the second photographing means 2nd body image taken by (c) Side view (a) showing the state where the user moves his / her hand (finger) in the moving space, image (b) of the second band, and image (c) showing the image of pixel analysis by the image analysis means Side view showing the shadow of a finger on the second band (a), image of the second band (b), and image when the second band is dark (c) Side view (a) showing a state where caps of different colors are put on a plurality of fingers and an image of the first belt (b) Side view showing a configuration including a plurality of video playback devices (a) and a diagram showing a pixel analysis image by video analysis means (b) The perspective view which shows the structure of the three-dimensional touch panel of 2nd Embodiment Side view of the 3D touch panel (a), front view from the user's side (b), and video taken by the third imaging means (c) Side view (a) showing a configuration in which the second belt is disposed on the upper side and the second reflecting mirror is disposed on the lower side, images (b) and image analysis of the first belt and the second belt captured by the third photographing means Figure (c) showing an image of pixel analysis by means Side view showing a state in which the moving space, the first belt and the second belt are installed outdoors A perspective view (a) and a longitudinal sectional view (b) showing the configuration of the second embodiment of the three-dimensional touch panel system A perspective view (a) showing an example of use of a three-dimensional touch panel system and a diagram (b) showing an image on an image display surface A perspective view showing the curved image display surface and the first belt (a) and the image of the first belt (b) Side view showing a state in which the first band and the second band are divided into three in the Z-axis direction, an image of the first band (b) and an image of the second band (c) Side view showing the second band divided into two in the Z-axis direction and the image of the second band (b) Side view (a) showing a state in which the second strip divided into two in the Z-axis direction is displayed on the video display device and an image of the second strip (b) The perspective view which shows the structure which attached the 1st reflective mirror and the 2nd reflective mirror so that angle adjustment was possible with respect to a flame | frame. The perspective view (a) showing the configuration when the first belt is divided into three in the Y-axis direction and the second belt is divided into three in the X-axis direction, and fixing and connecting means for each belt are shown. Figure (b) Side view (a) showing a modification of the first belt body and the second belt body, a diagram (b) showing a pixel analysis image by the video analysis means, and a diagram (c) showing a video on the video display surface The perspective view (a) which shows the structure which a projector projects the 1st belt and the 2nd belt on a plate, the front view (b) seen from the projector side, the image (c) of the 1st belt, and the 2nd belt Picture (d) Side view (a) showing a state in which the photographing means is housed inside a box having a slit and an image of the second belt (b) Side view showing a state in which cushioning material is arranged in the moving space Image (a) of the second belt containing noise and a diagram (b) showing the analysis image of the pixel when the image analysis means performs noise removal processing Image (a) of the second band and image (b) showing an analysis image of pixels when the image analysis means performs processing based on the absolute value of the RGB value Side view showing a configuration in which the tip of a writing instrument such as a brush or sign pen is used as the object to be measured by tilting the 3D touch panel sideways Side view showing a configuration in which cushioning materials are placed with the 3D touch panel lying down sideways Side view (a) and plan view (b) showing a configuration in which three video playback devices are arranged. Perspective views of the three-dimensional touch panel system in the embodiment (a) and (b) Diagram showing video on the video display surface Figures (a) to (d) showing the state of moving a finger Figures (a) to (c) showing the state of moving a finger

[First embodiment of non-contact type three-dimensional touch panel]
Hereinafter, embodiments of a non-contact type three-dimensional touch panel of the present invention (hereinafter, simply referred to as “three-dimensional touch panel”) will be described with reference to the drawings.

As shown in FIGS. 1 (a) to 1 (c), the three-dimensional touch panel 1 includes a moving space 10, a first belt 20, a second belt 30, a first photographing means 40, a second photographing means 50, and an image analyzing means. With 60. The three-dimensional touch panel 1 is arranged in an XYZ three-dimensional coordinate system in which the X-axis direction is the left-right direction, the Y-axis direction is the up-down direction, and the Z-axis direction is the front-back direction.
The moving space 10 is a range in which the object A to be measured is moved. Examples of the measurement object A include, but are not limited to, a user's hand, a fingertip, a tip of a pointing rod, and a pen tip. Hereinafter, the measurement object A will be described as a user's hand (finger).
The moving space 10 may simply have air, but a cushioning material 115 (see FIG. 21) having translucency and flexibility may be disposed as will be described later.

The first belt body 20 is a colored member disposed in the YZ plane along the moving space 10, and the second belt body 30 is a colored member disposed in the ZX plane along the moving space 10. That is, the first belt 20 is disposed on either the left or right surface of the four surfaces on the upper, lower, left, and right sides that form the moving space 10, and the second belt 30 is disposed on either the upper or lower surface.
Examples of the material of the first belt 20 and the second belt 30 include, but are not limited to, cloth, paper, plastic, metal, wood, and the like.
As the colors of the first band 20 and the second band 30, it is preferable to select a color that increases the color and contrast of the object A to be measured.

The first photographing means 40 is provided to face the first belt body 20 with the moving space 10 sandwiched in order to photograph the first belt body 20, and the second photographing means 50 is for photographing the second belt body 30. In addition, it is provided to face the second belt 30 with the moving space 10 in between. Examples of the first photographing means 40 and the second photographing means 50 include known video cameras.
The video analysis means 60 is to be measured by analyzing the first video 41 relating to the first belt 20 taken by the first photographing means 40 and the second video 51 relating to the second belt 30 taken by the second photographing means 50. It is provided to calculate the position of the object A on the YZ coordinate and the position on the ZX coordinate. In this embodiment, it is assumed that the video analysis means 60 is stored in the computer C.

Next, an operation algorithm of the three-dimensional touch panel will be described.
First, the first photographing means 40 photographs the first belt body 20, and the second photographing means 50 photographs the second belt body 30.
When the user puts the hand (finger) A into the moving space 10 as shown in FIG. 2 (a), the hand (finger) A is reflected in the first video 41 and the second video 51. As shown in FIGS. 2B and 2C, when the second video 51 is taken as an example, the video analysis means 60 divides the X-axis direction into X1, X2,... Xn, and the Z-axis direction into Z1, Z2,. The second video 51 is defined as an aggregate composed of n × n pixels. Then, the video analysis means 60 analyzes the color data of each pixel to identify the pixel (black portion) that overlaps the user's hand (finger) A.
The analysis of the pixel color data uses the RGB value that is the color data value of the pixel, the RGB value of the pixel (gray part) in which the second band 30 is reflected, and the user's hand (finger) A This is done by comparing the RGB value of the pictured pixels (black areas).
For example, when the color of the second band 30 is green (RGB value is R = 0, G = 255, B = 0), the video analysis means 60 acquires and compares the RGB value of each pixel, and the RGB value is Pixels that are the same as or close to (0,255,0) and different pixels are extracted. Thereby, the pixel in which the first band 20 and the second band 30 are shown and the pixel in which the user's hand (finger) A is shown are discriminated, and each of the user's hand (finger) A is shown. The pixel farthest (distant) in the Z-axis direction among the pixels is determined to be a pixel overlapping with the user's finger A, and the position of the finger A in the ZX direction is calculated by calculating the ZX coordinate of the pixel.
The same analysis is performed on the first band 20 to calculate the position of the hand (finger) A in the YZ direction, and thereby the XYZ coordinates of the finger A are calculated.

In the above algorithm, when the XYZ coordinates of the hand (finger) A are calculated, it is only necessary to compare the RGB values of the respective pixels. In addition to RGB values, CMYK values or the like may be used as color data values.
Also, the video analysis means 60 stores the positions of the first band 20 and the second band 30 in advance, and analyzes only the pixels in which the first band 20 and the second band 30 are reflected. For example, the processing speed can be increased compared to the case where the entire first video 41 and second video 51 are analyzed. The video analysis means 60 may store the positions of the first band 20 and the second band 30 in advance, and may analyze the entire pixels of the first video 41 and the second video 51. .
Another algorithm is to obtain the RGB values of all pixels when discriminating between the pixels in which the first band 20 and the second band 30 are shown and the pixels in which the user's hand (finger) A is shown. Instead of this, the RGB values of the pixels may be acquired and determined at regular intervals. In this case, the processing speed can be increased.

The color of the first belt 20 and / or the second belt 30 may be dark.
As shown in FIGS. 3 (a) and 3 (b), the shadow A ′ of the user's hand (finger) A is unintentionally caused by the indoor lighting L or the light of other equipment, etc., and the first belt 20 and / or There is a possibility that the image appears on the second band 30 and the color of the first band 20 and / or the second band 30 changes partially. This becomes noise when color data is analyzed by the video analysis means 60, and the XYZ coordinates of the user's hand (finger) A may not be detected accurately (that is, alias noise is generated due to shadows). Therefore, as shown in FIG. 3 (c), the first band 20 and / or the second band 30 should be dark (eg, black, R = 0, G = 0, B = 0 in RGB notation), etc. The shadow can not be seen or can be difficult to see. This eliminates the need to incorporate a shadow alias noise removal algorithm on the computer C side, and automatically removes the effects of shadow noise simply by extracting pixels with colors other than R = 0, G = 0, and B = 0. Thus, only the XYZ coordinates of the user's hand (finger) A without an alias can be acquired, and speedup can be realized.
Note that the object to be measured A whose coordinates are detected by the video analysis means 60 in one analysis is not limited to one. For example, as shown in FIGS. 4 (a) and 4 (b), by putting a cap A1 of a different color on a plurality of fingers A, or holding the writing instrument A2 on the opposite hand, the tip of the writing tip of these cap A1 or writing instrument A2 Coordinates can be detected simultaneously. Further, a color different from the surrounding color (see a triangular portion) may be attached to a part of the first belt 20 and / or the second belt 30.

[First embodiment of non-contact type three-dimensional touch panel system]
A first embodiment of a non-contact type three-dimensional touch panel system 100 (hereinafter sometimes simply referred to as “three-dimensional touch panel system”) will be described.
As shown in FIGS. 1 (a) to 1 (c), a three-dimensional touch panel system 100 according to the present embodiment includes the three-dimensional touch panel 1, a video playback device 101, and video processing means 102.
The video reproduction device 101 is arranged at a position where the user can visually recognize the video through the moving space 10. Examples of the video playback device 101 include a television, a personal computer, and a monitor. The video data to be played back is processed by the computer C.
Based on the position on the YZ coordinate and the position on the ZX coordinate of the measurement object A calculated by the video analysis means 60, the image processing means 102 determines which position on the image the user has moved the measurement object A. It is provided to calculate whether the position corresponds to the position and to change the image according to the position of the measurement object A on the image. In the present embodiment, it is assumed that the video processing means 102 is stored in the computer C.

When the user moves the hand (finger) A in the moving space 10 while watching the video of the video playback device 101, the video analysis means 60 calculates the XYZ coordinates of the hand (finger) A, and the video processing means 102 ) Change the image by calculating the position of the XYZ coordinates of A corresponding to the image being played.
As shown in FIGS. 5A and 5B, the three-dimensional touch panel system may include a plurality of video playback devices 101. The video may be a pointer P. In this case, the pointer moves on the video display surface 106 in accordance with the movement of the user's hand (finger) A.

[Second Embodiment of Non-contact 3D Touch Panel]
Next, a second embodiment of the three-dimensional touch panel will be described, but the same reference numerals are given to portions having the same configuration as the first embodiment, and the description thereof is omitted.
As shown in FIGS. 6, 7 (a) to (c) and FIGS. 8 (a) to (c), the three-dimensional touch panel of the present embodiment includes a first reflecting mirror 70 and a second reflecting mirror 71. It is characterized in that it includes a point and one photographing means (third photographing means 80).
The first reflecting mirror 70 is disposed at a position facing the first belt 20 with the moving space 10 in between, and the second reflecting mirror 71 is disposed at a position facing the second belt 30 with the moving space 10 in between. . Specifically, a first reflecting mirror 70 is fixed to the left side of a square frame 72 surrounding the upper, lower, left and right sides of the moving space 10, and a second reflecting mirror 71 is fixed to the upper side. The reflecting surface of the first reflecting mirror 70 faces the moving space 10 side and rotates forward when viewed in plan. The reflecting surface of the second reflecting mirror 71 rotates toward the front side when viewed from the side and viewed from the side. Light weight may be achieved by using thin film type mirrors or the like as the first reflecting mirror 70 and the second reflecting mirror 71.

The third photographing means 80 is in front of the frame 72 and is capable of photographing the first band image 81 reflected by the first reflecting mirror 70 and the second band image 82 reflected by the second reflecting mirror 71. It is fixed to. FIG. 7 (c) shows an image taken by the third photographing means 80. The first belt 20 and the user's hand (finger) A are shown on the left side of the image, and the second image is shown on the upper side of the image. The band 30 and the user's hand (finger) A are shown.
The first band image 81 and the second band image 82 photographed by the third photographing means 80 are analyzed by the video analysis means 60, and the position on the YZ coordinate and the position on the ZX coordinate of the hand (finger) A are determined. Calculated.
In the configuration of the three-dimensional touch panel shown in FIG. 8 (a), paying attention to an arbitrary Z coordinate as shown in FIGS. 8 (b) and (c), both the first belt body 20 and the second belt body 30 Obtain the XY coordinates of the user's hand (finger) A on the Z coordinate by analyzing the pixel whose color is different from the color of the band, that is, the pixel where the hand (finger) A overlaps the band can do.
Note that the first reflecting mirror 70 and the second reflecting mirror 71 may be fixed to a position along the moving space 10 with a column or the like without being fixed to the frame 72.
In the case of the configuration of the present embodiment, the third photographing means 80, the video reproduction device 101, and the like can be installed at positions away from the moving space 10, the first belt body 20, and the second belt body 30. Therefore, as shown in FIG. 9, the third photographing means 80, the video playback device 101, the computer C, etc. are installed indoors, and the moving space 10, the first belt body 20, and the second belt body 30 are installed outdoors. You can also. The user is outdoors and operates the three-dimensional touch panel while viewing the image of the indoor image reproducing device 101 through a transparent plate such as the window glass G.

[Second Embodiment of Non-contact 3D Touch Panel System]
Next, a second embodiment of the three-dimensional touch panel system will be described, but the same reference numerals are given to portions having the same configuration as the three-dimensional touch panel and the three-dimensional touch panel system, and the description thereof is omitted.
As shown in FIGS. 10A and 10B, the non-contact type three-dimensional touch panel system of the present embodiment is characterized in that a projector 103 is used as the video playback device 101. FIG.
In this case, the transmissive projector screen 104 is attached so as to cover the opening of the frame 72. By using the projector 103, the enlargement of the video can be realized.
FIGS. 11 (a) and 11 (b) show a state in which the figure of a woman wearing a costume is projected on the transmissive projector screen 104. FIG. For example, if the user touches the skirt with his hand, the video processing means 102 is changed to a skirt with a different design, which is useful when the user considers the coordination of the costume.
A transparent protective plate 105 is attached to the back surface of the transmissive projector screen 104, and the transmissive projector screen 104 is affixed to the protective plate 105 to prevent wrinkles and the transmissive projector screen 104 may be damaged by the user's hand. Can be prevented.
As another configuration example of the non-contact type three-dimensional touch panel system, as shown in FIGS. 12 (a) and 12 (b), the video display surface 106 is configured by a curved surface, and the first belt body 107 and the second belt body ( (Not shown) may be shaped according to the curvature. Note that the present invention can also be applied to a flexible video display surface called a so-called flexible display.

[Third embodiment of non-contact type three-dimensional touch panel]
Next, a third embodiment of the three-dimensional touch panel will be described, but the same reference numerals are given to portions having the same configuration as the above-described embodiments, and the description thereof is omitted.
As shown in FIGS. 13 (a) to (c), in the three-dimensional touch panel of the present embodiment, the first belt body 108 and the second belt body 109 are divided into two or more in the Z-axis direction (108a to 108c, 109a to 109c), and is characterized in that the position of each divided Z-axis direction is variable.
For example, as shown in FIGS. 14 (a) and 14 (b), the second band 30 is closer to the user as the second band 109a, and the far side is the second band 109b, and each has a different color. deep. The video analysis means 60 detects when the user's hand (finger) A overlaps the second band 109a closer to the user and when the hand (finger) A overlaps the second band 109b. To do.
For example, when the hand (finger) A overlaps the second band 109a, the power of the 3D touch panel is turned “ON”, and when the hand (finger) A overlaps the second band 109b, the user The video processing means 102 may determine that the contact has been made in a pseudo manner.
Also, the required distance for the user to move the hand (finger) A to the second band 109b by changing the relative distance in the front-rear direction (Z-axis direction) between the second band 109a and the second band 109b. Therefore, it is possible to adjust the sensitivity when the video processing means 102 determines whether or not the user has touched the video being played back by the video playback device 101. For example, if the relative distance between the second band 109a and the second band 109b is shortened, the sensitivity can be improved, and if the relative distance is increased, the sensitivity can be decreased.

For the second band 109b, instead of acquiring the RGB values of all the pixels, the X coordinate of the pixel where the hand (finger) A overlaps with the first band 109a is calculated, and the vicinity of this X coordinate If only the RGB values of the pixels existing in are acquired, the processing speed can be increased. Alternatively, the processing speed can be increased by acquiring and determining the RGB values of the pixels at regular intervals on the second band 109b.
Rather than physically dividing the second band 109, a mechanism for changing the relative distance between the

second bands

109a and 109b by displaying on the video display device 130 as shown in FIGS. 15 (a) and 15 (b) It may be. In this case, the color of the

second belt bodies

109a and 109b can be changed according to the surrounding environment such as the brightness and position of the illumination L, and the effect that the accuracy of the three-dimensional touch panel can be improved, A point is that the shadow of the user's hand (finger) A is difficult to be seen on the body 109. For example, if the user has a dark skin due to sunburn, etc., the advantage of preventing noise caused by shadows while increasing the accuracy of coordinate acquisition by using the white color opposite to black as the color of the

second band

109a, 109b The same effect as described above can also be obtained when the first belt body 108 is divided into two or more in the Z-axis direction and the divided positions in the Z-axis direction are made variable.

As indicated by arrows in FIG. 16, it is preferable that the first reflecting mirror 70 and the second reflecting mirror 71 are attached to the frame 72 so that the angle can be adjusted. In this case, the first reflecting mirror 70 and the second reflecting mirror 71 can be adjusted to the optimum angle (ideally 45 degrees according to the physical law that makes the incident angle and the reflection angle equal) toward the third photographing means 80. Note that the attachment positions of the first reflecting mirror 70 and the second reflecting mirror 71 to the frame 72 are not particularly limited.
When the first band image 81 and the second band image 82 captured by the third imaging unit 80 are distorted into a trapezoid or the like, the video analysis unit 60 may correct the image to a square or the like.
In order to prevent erroneous recognition by the video analysis means 60, it is preferable that the frame 72 has a color different from the colors of the first band 20 and the second band 30.

Fig. 17 (a) shows the configuration when the first belt 110 is divided into three in the Y-axis direction (110a to 110c) and the second belt 111 is divided into three in the X-axis direction (111a to 111c). It is an example. In this case, for example, when the user's hand (finger) A reaches the first belt 110a, it touched weakly (when turned ON), and when it reached the middle first belt 110b (when turned ON), it touched normally. When the first band 110c is reached (when turned ON), a ternary signal that makes sense such as touching it strongly is sent to the computer C together with the XY coordinate data, and the video processing means 102 reflects it in the video Is also possible. Furthermore, since the first belt bodies 110a to 110c are visible to the user, the user can visually recognize the first belt bodies 110a to 110c and determine how far the 3D touch panel reacts when the hand (finger) A is moved in the Z-axis direction. Can be used as expected. As shown in FIG. 17 (b), the belt bodies 110a to 110c and 111a to 111c are detachably fixed to the frame 72 by known means such as suckers and magnets, and the belt bodies are also connected by known means such as hooks. do it.
The first bands 110a to 110c are not necessarily different colors, but two first bands that are continuous in the Z-axis direction, that is, the

first bands

110a and 110b or the

first bands

110b and 110c. In order to prevent erroneous recognition by the video analysis means 60, it is preferable to use different colors.

[Fourth embodiment of non-contact 3D touch panel]
Next, a fourth embodiment of the three-dimensional touch panel will be described, but the same reference numerals are given to portions having the same configuration as the above-described embodiments, and the description thereof is omitted.
As shown in FIGS. 18 (a) to (c), the three-dimensional touch panel according to the present embodiment includes the

first belt

110A and 110B and the

second belt

111A and 111B in the middle of the first belt 110B. It is characterized in that the part and the middle part of the second band 111B are shifted in the direction close to the user. In this case, the image processing means 102 is used when the user's hand (finger) A overlaps both the middle part of the first belt 110B and the middle part of the second belt 111B, that is, the black part of FIG. 18 (b). It is recognized that the user touches the three-dimensional touch panel only when the user's hand (finger) A exists in the painted area.
Then, in the video playback device 101, if the button B is displayed at the position of the video represented by the XY plane as shown in FIG. A three-dimensional touch panel system can be constructed that determines that button B has been pressed simply by moving hand (finger) A to a location.
As another application example, for example, a device for teaching or playing such as a touching 3D three-dimensional map in which a video of a mountain or contour line is recorded on the video playback device 101 and the user can touch the mountain can be cited.

Further, as shown in FIGS. 19 (a) and 19 (b), a projector 103 is used as the video playback device 101, and the projector 103 projects light as the first belt 20 and the second belt 30 onto the plate 112. You may make it the structure to carry out.
In this case, by changing the light projected from the projector 103 with the computer C, the first belt 20 and the second belt 30 can be easily divided and resized, and the projector 103 can be used to make a large image. Can be realized. Further, the manufacturing cost can be suppressed by reducing the number of components of the three-dimensional touch panel.

[Fifth embodiment of non-contact 3D touch panel]
Next, a fifth embodiment of the three-dimensional touch panel will be described, but the same reference numerals are given to portions having the same configuration as the above-described embodiments, and the description thereof is omitted.
As shown in FIGS. 20 (a) and (b), in the three-dimensional touch panel of the present embodiment, the photographing means (40, 50, 80) is housed inside a box 114 having a slit 113, It is characterized in that the image is taken through the slit 113.
The width of the slit 113 is not particularly limited as long as the first band 20 and the second band 30 can be photographed. When the photographing means (40, 50, 80) is installed outdoors, it prevents extraneous noise from surrounding light sources, such as lighting L such as electric lights and direct sunlight, and the lens of the photographing means (40, 50, 80). The width of the slit 113 is preferably as narrow as possible in order to prevent fouling or to hide the existence of the photographing means (40, 50, 80) from the user as much as possible. Even when the photographing means (40, 50, 80) is installed indoors, the light from the lighting device or the projector 103 can directly enter the lens of the photographing means (40, 50, 80) by housing in the box 114. Can be prevented, and malfunction can be prevented.

[Sixth embodiment of non-contact 3D touch panel]
Next, a sixth embodiment of the three-dimensional touch panel will be described, but the same reference numerals are given to portions having the same configurations as those of the above embodiments, and the description thereof is omitted.
As shown in FIG. 21, the three-dimensional touch panel according to the present embodiment includes a cushioning material 115 having translucency and flexibility, and is characterized in that the cushioning material 115 is disposed in the moving space 10.
The cushion material 115 is formed in a hollow shape by a material having translucency and flexibility such as soft vinyl chloride. The cushion material 115 is filled with a substance such as a gas such as air, a transparent liquid such as water, or a gel having transparency.

[Seventh embodiment of non-contact 3D touch panel]
Next, a seventh embodiment of the three-dimensional touch panel will be described, but the same reference numerals are given to portions having the same configurations as those of the above-described embodiments, and the description thereof is omitted.
As shown in FIGS. 22 (a) and 22 (b), the three-dimensional touch panel of the present embodiment is characterized in that the video analysis means 60 performs noise removal.
Specifically, as described above, the video analysis means 60 is video, that is, the first video 41 shot by the first shooting means 40, the second video 51 shot by the second shooting means 50, and the third shooting means 80 shot. When analyzing the image of the first band 20 and the image of the second band 30 captured by the third imaging means 80, pixels having a color different from the color of the band are extracted. At this time, noise N may be reflected in each image, so the image analysis means 60 is used when the pixel is not continuous more than a certain number in any of the X, Y and Z axis directions. These pixels are judged as noise, and the position coordinates are not calculated. In FIG. 22 (b), two or more consecutive pixels in the Z-axis direction and two or more consecutive pixels in the X-axis direction are determined as noise. As described above, the image analysis means 60 performs noise removal, so that the accuracy of calculating the position coordinates of the object A to be measured can be improved.

[Eighth embodiment of non-contact 3D touch panel]
Next, an eighth embodiment of the three-dimensional touch panel will be described, but the same reference numerals are given to portions having the same configurations as those of the above embodiments, and the description thereof is omitted.
As shown in FIGS. 23 (a) and 23 (b), the three-dimensional touch panel of the present embodiment has an absolute value of the difference between the RGB values of two adjacent pixels when the video analysis means 60 analyzes the video. It is characterized in that the position on the measurement object A on the YZ coordinate and the position on the ZX coordinate are calculated based on the size. Specifically, the video analysis means 60 calculates the absolute value of the difference between the R values of two adjacent pixels G1 and G2, the absolute value of the difference between the G values, and the absolute value of the difference between the B values. When any one, or any two or more are above a certain value, it is determined that the measurement object A exists on the band, and the position of the measurement object A on the YZ coordinate and the ZX coordinate The position of is calculated.
Note that it is not necessary to calculate the difference between all adjacent pixels, and it may be limited only to the range in which the band exists from the viewpoint of increasing the processing speed. Further, as a method for further speeding up, it may be limited to only pixels in rows or columns with a predetermined interval.

[Third embodiment of a non-contact type three-dimensional touch panel system]
Next, a third embodiment of the three-dimensional touch panel system will be described, but the same reference numerals are given to portions having the same configuration as the three-dimensional touch panel and the three-dimensional touch panel system, and the description thereof is omitted.
As shown in FIG. 24, the non-contact type three-dimensional touch panel system of the present embodiment is characterized in that the three-dimensional touch panel is tilted sideways and the tip of a writing instrument such as a brush or a sign pen is used as the object A to be measured.
The first reflecting mirror 70, the second reflecting mirror 71, the first belt 20 and the second belt 30 are attached to a frame 116, and the frame 116 is supported by four legs 117.
In the case of the configuration of the present embodiment, the video played back on the video playback device 101 is plain at the initial stage, and the video processing means 102 sends the video playback device 101 such as letters and patterns in accordance with the movement of the tip A of the writing instrument. I will draw a picture.

[Fourth embodiment of non-contact 3D touch panel system]
Next, a fourth embodiment of the three-dimensional touch panel system will be described. The same reference numerals are given to the same parts as those of the three-dimensional touch panel and the three-dimensional touch panel system, and the description thereof will be omitted.
As shown in FIG. 25, the non-contact type three-dimensional touch panel system of the present embodiment is placed on a translucent plate 118 in a state where the three-dimensional touch panel is laid down on the transmissive projector screen 104. It is characterized in that a cushion material 119 having translucency and flexibility is disposed.
The plate body 118 is supported by four legs 120. A reflecting mirror 121 is disposed below the plate 118, and the light in the horizontal direction from the projector 103 is reflected in the vertical direction by the reflecting mirror 121 and projected onto the transmissive projector screen 104.
The user operates the three-dimensional touch panel by tracing the cushion material 119 with his / her hand while watching the image on the transmissive projector screen 104 while lying on the cushion material 119 in a facing direction.
As shown in FIGS. 26 (a) and (b), two or more video reproducing devices 101 may be arranged instead of the projector 103 and the reflecting mirror.

An embodiment of a non-contact type three-dimensional touch panel system will be described.
27 (a) and 27 (b), reference numeral 300 denotes a frame, 301 denotes a moving space, 302 denotes a first band, 303 denotes a second band, 304 denotes a first reflector, 305 denotes a second reflector, 306 Indicates third imaging means (ELECOM UCAM-C0220FENBK), 307 indicates video analysis means, 308 indicates a video reproduction device, and 309 indicates video processing means.
The first band 302 and the second band 303 are colored red (R: G: B = 240: 35: 35).
A tablet computer (with Windows 7 (registered trademark)) is used as the video playback device 308, and the video analysis device 307 and the video processing device 309 are stored in the video playback device 308.
The video display surface 310 of FIG. 28 shows a fish image, and the video analysis means 307 analyzes the position of the user's hand (finger) 311 as the object A to be measured, and the video display surface 310 shows ○ (sign P) The pointer is displayed. Reference numeral 312 denotes an image of the first belt 302 taken by the third photographing means 306, and 313 denotes an image of the second belt 303 taken by the third photographing means 306.
Note that the position of the user's hand (finger) 311 and the position of the pointer P do not match, but in this embodiment, in order to improve visibility and facilitate understanding, the user's hand (finger) 311 This is because the pointer is displayed by offsetting the XY coordinates of the position. Therefore, the position of the user's hand (finger) 311 and the position of the pointer P may be essentially matched and displayed on the video display surface 310.
The hand (finger) 311 is moved upward from the state shown in FIG. 29 (a) (0 seconds) to FIG. 29 (b) (0.35 seconds) and FIG. 29 (c) (0.7 seconds). When the hand (finger) 311 overlaps the fish image in (1.05 seconds), “touched !!” is displayed on the image display surface 106. After that, if the hand (finger) 311 is moved downward from the state of FIG. 30 (a) (2.80 seconds), the hand (finger) 311 is separated from the fish image at the time of FIG. 30 (b) (3.15 seconds). As a result, “touched !!” disappeared and the process ended in FIG. 30 (c) (3.50 seconds).
As described above, it has been demonstrated that the three-dimensional touch panel and the three-dimensional touch panel system of the present invention operate accurately.

The present invention relates to a touch panel having a simple device configuration, low manufacturing cost, and less likely to fail, and a touch panel system using this touch panel, and has industrial applicability.

A Object to be measured
A 'shadow
A1 cap
A2 Writing instrument
B button
C computer
G Window glass
G1, G2 pixels
L lighting
N noise
P pointer
1 3D touch panel
10 Moving space
20 Body 1
30 Second belt
40 First photography means
41 First video
50 Second shooting method
51 Second video
60 Video analysis means
70 First reflector
71 Second reflector
72 frames
80 3rd shooting method
81 Statue of the first belt
82 Second Belt Image
100 Non-contact 3D touch panel system
101 video playback device
102 Video processing means
103 Projector
104 Transmission type projector screen
105 Protection plate
106 Video display surface
107 1st Belt
108, 108a-108c 1st belt
109, 109a-109c 2nd zone
110, 110a-110c 1st belt
111, 111a-111c 2nd zone
112 plates
113 slit
114 box
115 Cushion material
116 frames
117 legs
118 plate
119 Cushion material
120 legs
121 reflector
130 Video display device
300 frames
301 Moving space
302 1st band
303 Second belt
304 1st reflector
305 Second reflector
306 Third imaging means
307 Video analysis means
308 Video playback device
309 Video processing means
310 Video display surface
311 Hand (finger)
312 Image of the first belt
313 Second Belt Image

Claims

In the XYZ three-dimensional coordinate system in which the X-axis direction is the left-right direction, the Y-axis direction is the up-down direction, and the Z-axis direction is the front-back direction,
A moving space that is a range in which the object to be measured is moved, and
A colored first strip disposed in the YZ plane along the movement space;
A colored second strip disposed in the ZX plane along the movement space;
A first reflecting mirror disposed at a position facing the first belt member across the moving space;
A second reflecting mirror disposed at a position facing the second belt member across the moving space;
Third imaging means for simultaneously capturing both the image of the first band reflected by the first reflecting mirror and the image of the second band reflected by the second reflecting mirror;
A non-contact type three-dimensional touch panel characterized by comprising: video analysis means for calculating a position on the YZ coordinate and a position on the ZX coordinate of the object to be measured by analyzing a video photographed by the third photographing means.
2. The non-contact type three-dimensional touch panel according to claim 1, wherein the first belt body and / or the second belt body is divided into two or more in the Z-axis direction.
3. The non-contact type three-dimensional touch panel according to claim 1, wherein a position of the first belt body and / or the second belt body in a Z-axis direction is variable.
The first band and / or the second band is divided into two or more in the longitudinal direction thereof, and the position of each divided part in the Z-axis direction is variable. The non-contact type three-dimensional touch panel according to any one of 3.
The non-contact type three-dimensional touch panel according to any one of claims 1 to 4, wherein a color of the first belt and / or the second belt is dark.
6. The non-contact type tertiary according to claim 1, wherein the first belt body and / or the second belt body is an image electrically displayed on a video display device. Former touch panel.
The non-contact type tertiary according to any one of claims 1 to 6, wherein the third photographing unit is housed in a box having a slit and photographs through the slit. Former touch panel.
It has a cushioning material with translucency and flexibility,
The non-contact type three-dimensional touch panel according to any one of claims 1 to 7, wherein the cushion material is disposed in the moving space.
When the video analysis unit analyzes the video shot by the third shooting unit, the video is divided into a plurality of pixels, and pixels having a color different from the color of the belt are in the X, Y, and Z axis directions. 9. The position coordinates are not calculated if the pixels are not continuous in a certain number in any of the directions, and the position coordinates are not calculated. Non-contact 3D touch panel.
When the video analysis unit analyzes the video shot by the third shooting unit, the video analysis unit divides the video into a plurality of pixels, and based on the magnitude of the absolute value of the difference between the RGB values of two adjacent pixels. 10. The non-contact type three-dimensional touch panel according to claim 1, wherein a position on the YZ coordinate and a position on the ZX coordinate of the measurement object are calculated.
A non-contact type three-dimensional touch panel according to any one of claims 1 to 10,
A video playback device disposed at a position where a user can visually recognize the video display surface through the moving space;
Based on the position on the YZ coordinate and the position on the ZX coordinate of the measurement object calculated by the image analysis unit, which position on the image display surface the position where the user moved the measurement object corresponds to A non-contact type three-dimensional touch panel system, comprising: a video processing unit that calculates a video displayed on the video display surface.
12. The non-contact type three-dimensional touch panel system according to claim 11, wherein the video is a pointer.
13. The non-contact type three-dimensional touch panel system according to claim 11, comprising two or more video playback devices.
14. The video display surface is formed of a curved surface, and the first belt body and / or the second belt body are curved corresponding to the curvature of the curved surface. A non-contact type three-dimensional touch panel system according to claim 1.
15. The non-contact type three-dimensional touch panel system according to claim 11, further comprising a translucent plate between the moving space and the video reproduction device.
In an XYZ three-dimensional coordinate system in which the X-axis direction is the left-right direction, the Y-axis direction is the up-down direction, and the Z-axis direction is the front-rear direction, YZ along the movement space that is the range in which the object to be measured moves A colored first strip in the plane, a colored second strip in the ZX plane,
Furthermore, a first reflecting mirror is disposed at a position facing the first belt body across the moving space, and a second reflecting mirror is disposed at a position facing the second belt body,
A third imaging means for simultaneously capturing both the image of the first band reflected by the first reflecting mirror and the image of the second band reflected by the second reflecting mirror;
A non-contact three-dimensional method characterized in that the image analysis means calculates the position on the YZ coordinate and the position on the ZX coordinate of the object to be measured by analyzing the image photographed by the third photographing means. Touch panel control method.
A program that causes a computer connected to the non-contact type three-dimensional touch panel to execute the control method of the non-contact type three-dimensional touch panel according to claim 16.
A recording medium on which the program according to claim 17 is recorded.