CN109358771B

CN109358771B - Touch screen

Info

Publication number: CN109358771B
Application number: CN201811298146.3A
Authority: CN
Inventors: 王厚生
Original assignee: Institute of Electrical Engineering of CAS
Current assignee: Institute of Electrical Engineering of CAS
Priority date: 2018-11-02
Filing date: 2018-11-02
Publication date: 2022-01-18
Anticipated expiration: 2038-11-02
Also published as: CN109358771A

Abstract

The invention relates to a touch screen, which is characterized in that a real-time scene image is obtained through a camera and is processed into a dot-matrix gray image in real time, then a gray signal of each pixel point of the dot-matrix gray image is converted into a current, voltage, infrared or light intensity signal dot matrix of a controller through matrix transformation and is output to the touch screen, the touch bar displacement of each pixel point of the display screen changes according to the output quantity of a corresponding control signal, the height of the surface formed by all dot-matrix touch bars on the touch screen changes along with the change of the output quantity of the corresponding control signal, the scene image is reproduced in a relief form, and along with the change of the scene, a corresponding 'gray' touch bar display part of a display device also changes along with the change of the display device in real time, so that a blind person can sense the real-time external scene change through the touch screen. The display screen of the invention has smaller size, and is similar to the size of a common mahjong, so that the blind can cover the whole touch screen by using the finger pulp of the thumb or other fingers.

Description

Touch screen

Technical Field

The invention relates to a touch screen for displaying visual images, belonging to the field of auxiliary identification equipment or auxiliary tools for people with visual disorders.

Background

The blind can recognize the braille or the surface information of the object by the touch, the traditional braille is statically written on various paper, and later people develop various electronic dynamic braille systems, the principle of which is to simulate the braille by controlling the motion of a dot-matrix touch bar, such as an electronic braille system panel developed by the institute of electrical and electronic engineers (IEE CAS)2008 in fig. 1. On the basis, a touch screen is developed. Fig. 2 is a large screen tactile image display developed by researchers in the united states and japan.

The existing touch display device utilizes an execution switch to complete touch bar array movement by a hydraulic system, and because the switch has only two working states of on and off, the touch bar generally has only two stations, namely the touch bar and an original position, the system displays braille or characters dynamically without problems, and when displaying images, only a 'black and white' mode is available, so that blind users are difficult to obtain touch images which have gray scale (brightness) or color difference feeling and are closer to the real world, and the current touch display is mostly static and cannot reflect changes of objects in external scenes in real time.

The related patents and literature reports at present:

burnt sun, etc., the university of Qinghua, the college of arts, the blind tactile image display Graile design research, decoration, 2016, (01), 94-96;

Vidal-Verdú,F.,Hafez,M.:Graphic Tactile Displaysfor Visually-Impaired P eople.IEEE Transactionsin Neural Systems and Reha bilitation En gineering，15(4),2007,119–130.)

liujun mark, etc., a mechanical dot matrix display, with the application number of 201310322754.4, the application date of 2013.07.29 national/province city, Beijing China (11), the publication number of 103413504A, the publication number of 2013.11.27, the main classification number of G09F 9/37(2006.01), the authorized bulletin number of 103413504B, the authorized bulletin number of 2015.09.23, the classification number of G09F 9/37(2006.01)

Xuluning, etc., a Braille electronic book with application number of 201310133140.1, application date of 2013.04.17, China/province city of Beijing (11), publication number of 104112372A, publication number of 2014.10.22, major classification number of G09B 5/00(2006.01), authorization notice number of 104112372B, authorization notice date of 2016.08.10, classification number of G09B 5/00 (2006.01); G09B 21/00(2006.01)

As can be seen from the prior literature reports and patents, the current touch screen technology for the blind mainly has the following defects:

(1) the size of the screen is similar to that of a display screen of a computer at present, two hands are required for touch, the blind can have the information concept of one screen only by touching the whole screen once, the speed of acquiring the information is relatively slow, and the screen cannot be carried about.

(2) The touch bars of the existing touch screen generally have only two working positions, namely the touch bars or the original positions, namely images of only black and white colors can be displayed, or corresponding braille or characters, while the images formed by objects in the nature are often colored or gray, the light and shadow change is continuous, and the existing touch screen has extremely poor reduction degree on real images;

(3) the display screen can only display the pre-stored content, generally used for reading the display, can only obtain the content input into the storage unit of the display in advance in a fixed occasion, has low information refreshing speed, updates a screen image from a few seconds to dozens of seconds, and cannot obtain the peripheral real-time information.

Disclosure of Invention

The technical problem of the invention is solved: the touch screen is small in size, can be covered by the finger abdomen of a thumb, can be carried about, can display a gray image in real time, can enable a person with visual disorder to acquire surrounding environment information at any time through touch, and plays a role in assisting vision.

The technical scheme of the invention is as follows: a tactile touch screen comprising: the system comprises a touch screen, a camera and an image data processing and control subsystem; the real-time scene image is obtained by the camera and is transmitted to the image data processing and controlling subsystem by a wired or wireless data transmission mode, the image data processing and controlling subsystem processes the image data into dot-matrix gray-scale images in real time, then the gray signal of each pixel point of the dot-matrix gray image is converted into a current, voltage, infrared or light intensity signal dot matrix through a control transformation matrix, a control signal is output to the touch screen, the touch bar displacement of each pixel point of the touch screen is changed according to the output quantity of the control signal, the height of the surface formed by all the dot-matrix touch bar arrays on the touch screen is changed along with the change of the touch bar displacement, and a scene image is reproduced in a relief form, and along with the change of the scene image, the corresponding gray-scale touch bar on the touch screen as a display component also changes in real time, so that the blind can sense the information of the real-time external scene image change through the touch screen.

The touch screen is a gray touch bar array display component, a dot-matrix touch bar array is manufactured by adopting electromagnetic force, magnetostrictive materials, piezoelectric ceramics, thermosensitive or photosensitive telescopic materials, and the working position of the touch bar is continuously variable from the initial position to the maximum displacement.

The control transformation matrix formula of the dot-matrix touch bar array is C, wherein C is KP^-1And K is a control coefficient matrix which is a characteristic matrix specific to each type of touch screen and needs to be solved, detected and determined according to actual structure size and physical parameters.

The formula of the control coefficient matrix K is as follows:

wherein k is_ijThe control mechanism of the tail of the contact rod in the ith row and the jth column independently acts on the influence coefficient of all the contact rods in the contact rod array, and the influence coefficient is a real value between 0 and 1, wherein i is 1 and 2. m and n are the number of rows and columns of the array.

The above-mentionedInverse P^-1The formula of (1) is as follows: PP (polypropylene)^-1＝E，P^-1For the inverse of the relief image to be displayed, P is the matrix of the image to be displayed and E is the identity matrix, the inversion of which can be referred to a professional book.

The invention processes the image obtained by the camera and the like through data, and displays the relief image corresponding to the image gray information on the touch screen in real time by controlling the touch bar array of the touch screen. The camera of the system is worn on the head of the visually impaired people, the touch screen is held in the hand, peripheral visual information can be acquired in real time through touch, and the problem of life self-care and even work conspiracy of visually impaired people is solved.

Compared with the prior art, the invention has the advantages that:

(1) the volume and the weight are small, the mahjong piece is similar to a mahjong piece, the whole touch screen can be covered by the finger abdomen of the thumb, the information acquisition speed is high, the mahjong piece can be conveniently carried about, and the mahjong piece belongs to wearable auxiliary equipment;

(2) the height of the touch bar is continuously variable, the displayed image is a gray image, and the degree of restoration of external information is higher;

(3) the blind person can sense the change of the surrounding environment condition at any time.

Drawings

FIG. 1 is an electronic Braille system developed by IEECAS;

FIG. 2 is a tactile touch screen developed in the United states and Japan; wherein a is a tactile touch screen developed by NIST in the united states, and b is a tactile display developed by KGS corporation of japan in cooperation with national institute of development (NASDA) of japan; c is a tactile touch screen developed by NHK of japan;

FIG. 3 is a schematic diagram of the principles of the present invention;

FIG. 4 is a schematic diagram of an electromagnetic single pixel of the present invention;

FIG. 5 is a schematic diagram of a magnetostrictive, photostrictive, and thermo-strictive single pixel of the present invention;

FIG. 6 is a schematic diagram of a piezoelectric telescopic single pixel of the present invention;

FIG. 7 is a top view of a matrix of individual pixels arranged in rows and columns of a tactile touch screen embodiment of the present invention.

Detailed Description

The invention is further described below with reference to the accompanying drawings and the detailed description.

As shown in FIG. 3, the touch screen capable of displaying gray scale images in real time comprises corresponding image acquisition (camera), image data processing and control subsystems and a gray scale touch bar array display component, namely the touch screen.

The invention utilizes electromagnetic force, ferromagnetic telescopic material, piezoelectric ceramic or photosensitive telescopic material and the like to obtain a real-time scene image through a camera and process the real-time scene image into a dot-matrix gray image, then the gray signal of each pixel point of the dot-matrix gray image is converted into a current, voltage or light intensity signal array of a controller through matrix transformation and is output to a touch screen, the touch bar displacement of each pixel point of the touch screen changes according to the corresponding control signal output quantity, the surface formed by all dot-matrix touch bars on the touch screen also changes along with the change of the touch screen, the scene image is reproduced in a relief form, and along with the change of the scene, the corresponding gray touch bar display part of the display device also changes along with the change of the scene in real time, so that a blind person can sense the real-time external scene change through the touch display. The display screen of the invention has smaller size, which is similar to the size of a common mahjong, so that the blind covers the whole touch screen by using the finger pulp of the thumb or other fingers.

The touch screen is composed of a plurality of display pixels, the pixels are parallel-arranged long and thin bars with the same length and section size, namely touch bars, the touch bars can move up and down under the action of a control mechanism at the tail part, the up and down movement amount of the image data processing subsystem is related to the gray level or color signals of the acquired video image according to the real-time image acquired by the camera, for example, when the video image is pure black or pure white, namely 8-bit tricolor function values RGB (0,0,0), all the touch bars in the array are located at the initial position; RGB (255 ), all the touch bars in the array are in the highest position; the middle position of the touch bar is linearly corresponding to or obtains a gray value between pure white and pure black according to a set visual brightness interpolation algorithm. When the image conversion processing and output subsystem converts the image signals acquired by the camera into grayscale relief effect images in real time, the touch bars at corresponding array positions in the touch screen are controlled in real time to adjust the height of the touch bars so as to correspond to the pixel grayscale value of the images, so that the relief effect images acquired by the image conversion processing and output subsystem can be restored and displayed on the touch screen, and a blind person can timely sense the acquired scene in the current camera according to the touch sense.

When the image conversion processing and output subsystem controls the height position of each touch bar by adjusting the strength of signals such as electricity, magnetism, light (including invisible light which can act on photosensitive materials such as infrared and ultraviolet rays) and the like of a control mechanism at the tail part of the corresponding touch bar in the touch screen according to the gray value of pixels of a gray image. It should be noted that the control mechanism at the tail of each touch bar not only acts on the high and low positions of the connected touch bar, but also affects the high and low positions of the adjacent touch bars which are separated by an even farther distance, therefore, the value of the control quantity (current or electric field intensity, magnetic field intensity, light intensity, etc.) in the control mechanism at the tail of each touch bar does not correspond to the gray value of the image linearly, an initial gray image array matrix of m rows and n columns is set as a,

wherein a is_ijI is the gray value of the pixel in the array, 1,2,. m, j is 1,2,. n,

and has a_ijConst, that is, a is a constant matrix, that is, in an initial state, the heights of the touch bars are consistent, and the touch screen appears to be a plane;

when the control mechanism of the tail of the touch bar in the ith row and the jth column acts alone, the matrix of the influence on the image is as follows:

wherein k is_ijThe control unit acting alone as the tail of the feeler in row i and column j has an influence coefficient on all the feelers in the feeler array, which is typically a real value between 0 and 1,

i

1,2,. m,

j

1,2,. n, the height of the feeler varies, as follows:

AK(ij)＝P_ij (3)

P_ijthe display result height matrix of the current touch bar array, namely the touch screen after the control mechanism of the touch bar tail part of the ith row and j column is acted, because the control mechanism of the touch bar tail part of the ith row and j column not only acts on the touch bar of the ith row and j column, but also acts on the up-down height displacement of the touch bar near the periphery of the ith row and j column, which is the most principle difference between displaying a gray image and a black-and-white mode.

When all the control mechanisms at the tail of the touch bar are in action, the following steps are provided:

p is the final display result height matrix of the tactile touch screen, let:

then there are:

AK＝P (6)

the K is called a control coefficient matrix, the control coefficient matrix is a characteristic matrix specific to each type of touch screen, and the control coefficient matrix needs to be solved and detected according to actual structure size and physical parameters.

In practical engineering, it is known that image data to be displayed by a touch screen, that is, data P of a relief image output by an image conversion processing and output subsystem, needs to be inverted to obtain a value of a control quantity in a control mechanism array at the tail of a touch stick, and then equation (6) needs to be transformed:

AKP^-1＝PP^-1 (7)

P^-1in order to display the inverse matrix of the relief image, the right side of the above equation becomes the unit matrix, and the control coefficient matrix for controlling the value of the mechanism array quantity:

C＝KP^-1 (8)

that is, the desired image P can be displayed in relief on the tactile touch screen by sending a corresponding controlled amount of C matrix signals to the control mechanism at the tail of each touch bar.

Note here that C, K, P and P^-1The touch bar tail control mechanism is a matrix, for example, C, and is specifically a touch bar tail control mechanism of the ith row and the jth column, namely the value of C (ij).

Detection and determination process of control coefficient matrix:

when the control mechanism at the tail of the touch bar in the ith row and the jth column acts alone, the touch bar in the corresponding ith row and the jth column is displaced by P_ijThis displacement can be measured with a ruler or other displacement measuring instrument, in which case the control variable in the tail control means, for example the current of the solenoid 5 in fig. 4, is C_ijThis value can be measured by an ammeter or the like. Then P_ijAnd C_ijHas a ratio k therebetween_ij，k_ij＝P_ij/C_ij. Now, if a certain touch bar is lifted and other touch bars are not moved on the touch screen, a convex point is displayed, and the current C at the tail part of the control device of the ith row and the jth column touch bar is touched by the blind person to control the tail part of the control device of the ith row and the jth column touch bar_ijThe expected result cannot be obtained because the invention aims to pursue that the size and the volume of the touch screen are small enough, the touch bars are close to each other, the tail control device is close to each other, the magnetic coil always has magnetic leakage, the magnetic coil 5 of the ith row and the jth column tail control mechanism not only raises the ith row and the jth column, but also raises the adjacent ith row, the jth +1 column, the ith row, the jth-1 column, the ith-1 row, the jth column, the ith +1 row, the jth column and even more farther touch bars to a certain extent, and the raising displacement is smallerIn P_ijThe amount of elevation of these feelers, which is less influenced the further the distance, can also be measured by a measuring instrument such as a ruler, and the amount of current C in row i and column j_ijObtaining a ratio of k of all the touch bars_ijAnd obtaining an influence coefficient matrix k (ij) of the ith row and j column touch bar control mechanisms on the whole touch screen. The adjacent touch bars have corresponding influence coefficient matrixes, in order to correctly display only one convex point, the control device at the tail part of the adjacent touch bar needs to be electrified with reverse current to offset the raised amount, and the reverse same current influences the adjacent touch bars, so that the display displacement control of each touch bar actually influences the control current of the tail control device of all the touch bars of the whole touch display, and the process is also the working principle derivation process of the formulas (4) to (7).

By analogy, the control principle of each pixel touch bar is the same, and the displacement P of each touch bar is controlled_ijAccordingly, a gray-scale image can be displayed.

Of course, for each touch screen of the same model, the control coefficient matrix is measured in advance after the sample is manufactured, stored in the algorithm of the image processing and control module, and directly and quickly called when the image is displayed.

As shown in fig. 3, the camera collects scene image data and transmits the scene image data to the image data processing and controlling module through a data line, the module processes the image data into a row-column lattice first to form a matrix and extract gray value data, and converts the image data into a control signal matrix according to the conversion process of the formulas (1) to (7), and controls the touch bar array action of the touch screen to form a real-time relief image. The touch screen has a very small area and is basically covered by the thumb and the finger belly, so that a person with visual impairment can sense the environmental visual information at any time through touch.

Fig. 4 is a schematic structural diagram of an electromagnetic single pixel in the single pixel embodiment 1 of the present invention. As shown in fig. 4, wherein 1 is a touch bar, the touch bar moves up and down along the hole of the guide plate 3 under the combined action of the spring 2 and the electromagnetic coil 5 fixed on the fixed plate 6, the guide rod 4 is made of ferromagnetic material, the displacement of the up and down movement is determined by the current in the electromagnetic coil 5, the current is the control quantity C (ij) of a single electromagnetic pixel, and the image data processing and control module sends the specific value of the ith row and the jth column of the control quantity matrix C to the electromagnetic coil 5 in each specific pixel through a control data transmission line to control the up and down displacement of the touch bar 1.

Fig. 5 is a schematic structural diagram of a single pixel of magnetostrictive, photosensitive telescopic, and thermosensitive telescopic embodiments 2 of the present invention, in which 1 is a touch rod, the telescopic rod 7 is fixed on the substrate 9, the telescopic rod 7 can be made of different response materials, 8 is a signal generator, and the specific type of the signal generator varies with the material of the telescopic rod 7: when the telescopic rod 7 is made of magnetostrictive materials, the signal generator 8 is an electromagnetic coil and is used for generating a magnetic field, and the telescopic rod 7 has different size and length changes due to different magnetic field strengths; when the telescopic rod 7 is made of photosensitive telescopic materials, the signal generator 8 is an LED light source, and different illumination intensities cause the length size change of the telescopic rod 7; when the telescopic rod 7 is made of a thermo-sensitive telescopic material, the signal generator 8 is an infrared generator (thermal radiation) for generating different infrared intensities to cause the size change of the telescopic rod 7.

Fig. 6 is a schematic diagram of a single pixel embodiment 3 of the present invention, which is a piezoelectric telescopic single pixel, wherein 1 is a touch bar, a telescopic rod 7 made of piezoelectric material is fixed on a substrate 9, a voltage lead 10 is connected on the telescopic rod 7 as an electrode, in this case, the substrate 9 is grounded, and as another electrode, when a non-conductive voltage is applied to the voltage lead, the length dimension of the telescopic rod 7 is changed.

In

embodiments

2 and 3, the control quantities c (ij) are the magnetic field intensity, the light intensity, the heat radiation intensity and the voltage, respectively, and the image data processing and control module controls the up and down displacement of the touch bar 1 by controlling the data transmission line to send to the signal generator 8 in each specific pixel or the voltage lead 10.

FIG. 7 is a top view of a touch screen embodiment of the present invention in a matrix of individual pixels arranged in rows and columns, with the touch bar 1 of each pixel seen in plan view, all of the touch bars being secured to a substrate 9 or a mounting plate 6 by telescoping rods 7 (hidden from view). The 16 × 12 matrix drawn in the figure can also be a matrix with other rows and columns, theoretically, the smaller the protruding area of the touch bar 1 of a single pixel is, the more the number of single pixels can be arranged in the display screen, the more exquisite the relief image can be displayed, and the larger the information amount can be obtained by touch.

Claims

1. A tactile touch screen, comprising: the system comprises a touch screen, a camera and an image data processing and control subsystem; the real-time scene image is obtained by the camera and transmitted to the image data processing and controlling subsystem in a wired or wireless data transmission mode, the image data processing and controlling subsystem processes the image data into dot-matrix gray-scale images in real time, then the gray signal of each pixel point of the dot-matrix gray image is converted into a current, voltage, infrared or light intensity signal dot matrix through a control transformation matrix, a control signal is output to the touch screen, the touch bar displacement of each pixel point of the touch screen is changed according to the output quantity of the control signal, the height of the surface formed by all the dot-matrix touch bar arrays on the touch screen is changed along with the change of the touch bar displacement, and a scene image is reproduced in a relief form, along with the change of the scene image, the corresponding gray-scale touch bar on the touch screen as a display component also changes in real time, so that the blind can sense the real-time information of the change of the external scene image through the touch screen;

the touch screen is a gray touch bar array display part, a dot-matrix touch bar array is manufactured by adopting electromagnetic force, magnetostrictive materials, piezoelectric ceramics, heat-sensitive or photosensitive telescopic materials, and the working position of the touch bar is continuously variable from the initial position to the maximum displacement;

the control transformation matrix formula of the dot-matrix touch bar array is C, and C is KP^-1K is a control coefficient matrix which is a characteristic matrix specific to each type of touch screen and needs to be solved, detected and determined according to actual structure size and physical parameters; p^-1The inverse matrix is the inverse matrix which is required to display the relief image in the scene image reproduced in the relief form;

the formula of the control coefficient matrix K is as follows:

wherein k is_ijThe control mechanism of the tail part of the contact rod of the ith row and the jth column independently acts on the influence coefficient of all the contact rods in the contact rod array, and the influence coefficient is a real value between 0 and 1, wherein i is 1 and 2, and.

2. A tactile touch screen according to claim 1, wherein: the inverse matrix P^-1The formula of (1) is as follows: PP (polypropylene)^-1＝E，P^-1To display the inverse of the relief image, P is the matrix of the image to be displayed and E is the identity matrix.