JP2019194798A - Control system and control program - Google Patents

Abstract

To share a drag operation available screen that a healthy person uses with a visually impaired person in such a manner that the impaired person is not confused by the operation.SOLUTION: In a normal mode, a scroll area a1 is provided in an operation screen 12b, and a plurality of icons i is arranged in a predetermined scroll direction in the scroll area a1 so as to be displayed in a scrollable manner in response to a drag operation by a user. In a visually impaired person mode, when the scroll area a1 displayed in the normal mode is displayed on the operation screen 12b, control is performed for the icon i in the scroll area not to move even if a drag operation is performed on the scroll area a1.SELECTED DRAWING: Figure 10B

Description

  The present invention relates to a control system and a control program that can be used by a visually impaired person.

  In general, a visually impaired person cannot operate with a normal touch panel. In order to solve such a problem, the technique disclosed in Patent Document 1 discloses a touch panel device that can switch between a normal mode and a visually impaired person mode. In this device, the user switches to the visually impaired person mode by wearing headphones or by operating a decision switch. In the visually impaired person mode, when the user touches the touch panel, the touch information is not notified to the control means, but only the touched position information is notified to the control means. The control means guides screen information corresponding to the position information by voice, and if the screen information is desired, the user turns on a determination switch different from the touch panel. When the determination switch is turned on, the touch panel device notifies the control unit of touch information so that the same operation as in the normal mode can be performed.

  In the technique disclosed in Patent Document 2, an input device using a user wearing a finger cap capable of wireless communication is disclosed, and the following processing is executed. When the user wearing the finger cap touches the touch panel, a Braille matrix signal corresponding to the operation type information of the coordinate position of the touch is transmitted to the finger cap. On the finger cap side, a three-dimensional Braille pattern corresponding to the Braille matrix signal is generated inside. The user recognizes the type of key by tracing the inner braille pattern.

JP 2004-271748 A JP 2000-172408 A

  In the field of information equipment, corresponding functions are increasing year by year, and the number of icons (buttons) for setting each function is also increasing. For this reason, it is difficult to display all icons corresponding to all functions at once on the operation screen for setting. As a technology to cope with this, a user interface (UI) for displaying a part of icons on a display screen by determining the order of icons and scrolling them according to a user's drag operation on the touch panel is widely used. It has been.

  However, in the techniques disclosed in the cited references 1 and 2, when the operation screen is scrolled and the operation screen is traced with a finger, the icon moves in accordance with the movement of the finger. Is only touching one icon at the beginning and end of tracing, and only the presence of the one icon can be recognized, and there is a possibility that an appropriate operation cannot be performed.

  The present invention has been made in view of the above circumstances, and an object of the present invention is to provide a control system and a control program capable of sharing a drag-operable operation screen used by a healthy person without being confused by a visually impaired person. And

  The above object of the present invention is achieved by the following means.

(1) A touch panel that displays an operation screen on the display surface and detects a touch on the display surface by a user;
A notification unit for notifying the visually impaired,
A display control unit that controls display contents of the operation screen, and executes a process associated with the icon in response to a selection operation on any of the plurality of icons displayed on the operation screen;
Notifying the user of the display content of the operation screen at the touch position on the touch panel via the notification unit, and performing the selection operation of the icon at the touch position by operating a determination switch different from the touch panel. A mode switching unit that switches between a visually impaired person mode and a normal mode for performing the selection operation of the icon by touching the icon on the touch panel;
The display control unit
In the normal mode, a scroll area is provided in the operation screen, a plurality of the icons are arranged in a predetermined scroll direction in the scroll area, and displayed in a scrollable manner by a drag operation by a user.
In the visually impaired person mode, when the scroll area displayed in the normal mode is displayed on the operation screen, the icon in the scroll area is not moved even if a drag operation is performed on the scroll area. Control system.

(2) A braille device worn by the user
The braille device is
A finger cover covering the user's fingertips;
The notification unit arranged inside the finger cover and capable of notifying braille information through touch to a visually impaired person;
The decision switch;
A communication unit that wirelessly communicates with the main body device provided with the touch panel,
The said display control part is a control system as described in said (1) which notifies the braille information regarding the display content of the said operation screen in the touch position to the said touch panel to a user by the said notification part.

  (3) Even in the visually impaired mode, when the drag operation is performed on the scroll area, the icon is moved when the drag operation is not performed by the Braille device. The control system described in.

(4) The braille device further includes a sensor unit that is disposed at a position corresponding to the belly of the finger of the finger cover and detects that the outside of the finger cover has contacted an object,
The display control unit determines whether the drag operation on the scroll area is performed by the Braille device based on a contact timing detected by the sensor unit transmitted from the Braille device and a touch timing detected by the touch panel. The control system according to (3), wherein it is determined whether or not.

(5) On the operation screen, a button for moving the icon is arranged in an area other than the scroll area,
The control according to any one of (1) to (4), wherein the display control unit moves the icon in the scroll area according to a selection operation on the button in the visually impaired person mode. system.

(6) The button is disposed at a position adjacent to the scroll area in the scroll direction,
The display control unit, in the visually impaired mode, in response to a touch operation on the button, the number of the icons that are hidden in the direction of the button, the number of the icons that are displayed, and display and hidden The control system according to (5), wherein the notification unit notifies a total number of the icons, which is a total of the icons that are present.

(7) The display control unit, in the visually impaired mode,
When a touch operation or a drag operation is performed on a boundary portion between two adjacent icons in the scroll area, the notification unit notifies information about an arrangement state of the icons in the scroll area. The control system according to any one of (1) to (6).

  (8) The mode switching unit switches to the visually impaired person mode according to establishment of wireless communication between the main body device and the Braille device. A control system according to any one of the above.

  (9) The mode switching unit switches from the visually impaired person mode to the normal mode in response to disconnection of wireless communication established between the main body device and the Braille device. ) Control system.

  (10) When the mode switching unit switches to the visually impaired person mode according to the establishment of wireless communication between the main unit and the braille device, or after switching The control system according to (8) or (9), wherein the operation mode is switched to the normal mode when a state in which the operation by the Braille device is not performed for a predetermined time or longer after the operation by the Braille device is performed. .

  (11) The mode switching unit responds to an input instruction from a user even when the mode is switched to the visually impaired mode in response to establishment of wireless communication between the main body device and the Braille device. The control system according to any one of (8) to (10), wherein the control mode is switched to the normal mode or the visually impaired person mode.

  (12) The control system according to any one of (1) to (11), wherein a common operation screen is displayed in the normal mode and the visually impaired person mode.

(13) An operation screen is displayed on the display surface, and is executed by a computer that controls a control system that includes a touch panel that detects a touch on the display surface by a user and a notification unit that notifies a visually impaired person of information. A control program,
A step (a) of controlling the display content of the operation screen and executing a process associated with the icon in response to a selection operation on any of the plurality of icons displayed on the operation screen;
Visual information for notifying the user of the display content of the operation screen at the touch position on the touch panel via the notification unit and performing the selection operation of the icon at the touch position by operating a determination switch different from the touch panel. (B) switching between a disabled person mode and a normal mode for performing the selection operation of the icon by touching the icon on the touch panel,
In the step (b), in the normal mode, a scroll area is provided in the operation screen, a plurality of the icons are arranged in a predetermined scroll direction in the scroll area, and scrollable by a drag operation by a user. Display
In the visually impaired person mode, when the scroll area displayed in the normal mode is displayed on the operation screen, the icon in the scroll area is not moved even if a drag operation is performed on the scroll area. A control program for causing the computer to execute processing.

  According to the present invention, in the normal mode, a scroll area is provided in the operation screen, and a plurality of icons are arranged in a predetermined scroll direction in the scroll area and displayed in a scrollable manner by a drag operation by the user. In the user mode, when the scroll area displayed in the normal mode is displayed on the operation screen, control is performed so that the icon in the scroll area is not moved even if a drag operation is performed on the scroll area. By doing in this way, the control system and control program which can share the operation screen which can perform the drag operation which a healthy person uses can be shared by a visually impaired person without being confused by operation can be provided.

It is a block diagram which shows the hardware constitutions of a main body apparatus. It is a block diagram which shows the hardware constitutions of a braille device. It is a perspective view which shows the state which touched the operation panel with the braille device. It is a perspective view which shows the external appearance of a braille device. It is a cross-sectional schematic diagram around a finger cover. It is an example of the operation screen displayed on a touch panel. It is a conceptual diagram which shows the operation screen (Braille image) for Braille corresponding to FIG. It is a flowchart which shows the control by the braille device side. It is a flowchart which shows the control by the side of the main body apparatus which concerns on 1st Embodiment. It is a subroutine flowchart of step S22 (normal mode) of FIG. 8A. It is a subroutine flowchart of step S23 (visually impaired person mode) of FIG. 8A. It is an example of the operation screen displayed on the operation panel in normal mode. It is a figure which shows the operation screen after dragging in FIG. 9A. It is an example of the operation screen displayed on the operation panel in visually impaired person mode. It is a figure which shows the operation screen after dragging in FIG. 10A. It is a subroutine flowchart which shows the control by the side of the main body apparatus which concerns on 2nd Embodiment. It is a subroutine flowchart which shows the control by the side of the main body apparatus which concerns on 3rd Embodiment. It is a figure explaining the function of the scroll button of an operation screen. It is a figure explaining the function of the scroll button of an operation screen. It is a conceptual diagram which shows the operation screen (Braille image) for Braille corresponding to FIG. 13B. It is a figure explaining the function of the scroll button of the operation screen in 4th Embodiment. It is a figure explaining the function of the boundary part between the icons of an operation screen. It is a figure explaining the function of the boundary part between the icons of an operation screen. It is a flowchart which shows the control by the side of the main body apparatus which concerns on 5th Embodiment. It is a flowchart which shows the control by the side of the main body apparatus which concerns on 6th Embodiment.

  Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings. In the description of the drawings, the same elements are denoted by the same reference numerals, and redundant description is omitted. In addition, the dimensional ratios in the drawings are exaggerated for convenience of explanation, and may be different from the actual ratios.

  FIG. 1 is a block diagram illustrating a hardware configuration of the main device 100, and FIG. 2 is a block diagram illustrating a hardware configuration of the braille device 200. 3A, 3B, and 4 are diagrams showing the configuration of the braille device 200 and the like. As shown in these drawings, the control system 500 includes a main device 100 and a braille device 200.

(Main unit 100)
The main device 100 is, for example, an MFP (Multifunction Peripheral). As shown in FIG. 1, the main device 100 includes a control unit 110, an operation panel 120, a storage unit 130, a printer unit 140, a scanner unit 150, a facsimile unit 160, a wireless communication unit 170, and a speaker 180.

  The control unit 110 includes a CPU (Central Processing Unit). The storage unit 130 includes an auxiliary storage device such as a RAM (Random Access Memory), a ROM (Read Only Memory), and an HDD (Hard Disk Drive). The ROM stores various programs and various data in advance, and the RAM temporarily stores programs and data as a work area. The CPU controls the entire main device 100 according to the program. The control unit 110 functions as the display control unit 111 and the mode switching unit 112.

  The operation panel 120 is also referred to as a touch screen, and includes an LCD 121 configured by a liquid crystal screen and a touch panel 122 that is superimposed on the LCD 121 and detects an operation of a user's finger.

  The touch panel 122 is, for example, a capacitive detection unit. When a user's finger touches the surface of the operation panel 120, the presence / absence of the finger touch and the contact position based on the change in the capacitance. Detect (coordinates). It also supports detection of simultaneous touch (multi-touch) with multiple fingers.

  The printer unit 140 forms an image on a sheet based on print data by a known electrophotographic method or inkjet method.

  The scanner unit 150 includes an original glass on which an original is placed, a light source that irradiates the original with light, an optical system such as a lens, and an optical element such as a CCD that generates image data from reflected light. The scanner unit 150 reads an image of a document placed on a document glass or a document transported by an automatic document transport device, and generates image data.

  The facsimile unit 160 includes a modem and a data access unit. The modem is connected to a data access unit and a communication line network such as a telephone line. The data access unit has a function of outputting a pulse signal corresponding to the destination telephone number. The facsimile unit 160 transmits image data to the facsimile machine of the other party through the communication line network.

  The wireless communication unit 170 includes a wireless communication interface such as BLE (Bluetooth (registered trademark) Low Energy), Bluetooth (registered trademark), IEEE 802.11, or IrDA. Further, as short-range wireless communication, a communication method such as FeliCa (registered trademark), ISO / IEC 14443 (MIFARE (registered trademark) communication method), ISO / IEC 18092 (NFC: Near Field Communication) may be used. . In addition to the above, the main body device 100 is a wired interface, such as a LAN connection interface based on a standard such as Ethernet (registered trademark) or FDDI, a serial interface such as USB or IEEE1394, or a parallel interface such as SCSI or IEEE1284. Etc. may be provided.

  The speaker 180 outputs sound. Specifically, the display content of the operation screen 12b displayed on the operation panel 120 functions as a notification unit that notifies the user such as the visually impaired with information by voice.

(Braille device 200)
As shown in FIGS. 2, 3A, 3B, and 4, the braille device 200 includes a control unit 210, a braille display unit 220, a selection switch 230 for selection operation, a contact sensor 240, and a wireless communication unit 250.

  As shown in FIGS. 3A and 3B, the braille device 200 is worn on the hand of a user such as a visually handicapped person, and includes a glove body 291, a finger cover 292 that covers the index finger in the worn state, and the glove body It is composed of a unit case 293 provided at a portion corresponding to the back of the hand of the portion 291. Braille device 200 is attached to the user's hand by a wrist band (not shown) of glove body 291.

  A Braille display unit 220, a determination switch 230, and a contact sensor 240 are provided at the tip of the finger cover 292, and a control unit 210 and a wireless communication unit 250 are provided inside the unit case 293. These are supplied with electric power by a secondary battery (not shown) provided in the unit case 293.

(Control unit 210, wireless communication unit 250)
The control unit 210 includes a CPU, a RAM, and a ROM. The wireless communication unit 250 communicates with the wireless communication unit 170 by wireless communication. These functions are the same as those of the control unit 110 and the wireless communication unit 170, respectively, and description thereof is omitted.

(Braille display unit 220)
The braille display unit 220 is disposed inside the finger cover 292 and displays (implements) a three-dimensional braille pattern that causes the user to feel the braille. The braille display unit 220 functions as a notification unit that notifies the visually impaired. FIG. 4 is a schematic cross-sectional view around the finger cover 292. As shown in the figure, the braille display unit 220 is provided with a large number of pins that make the fingertip feel a tactile sense, and the protruding amount of the pins in the height direction can be individually changed by the piezoelectric element. The pin density (resolution) is higher than a typical Braille pitch (2 to 3 mm), and text information and simple images can be displayed in addition to the Braille pattern.

  Hereinafter, an example of a braille pattern displayed on the braille display unit 220 will be described with reference to FIGS. 5 and 6. FIG. 5 is an example of the operation screen 12 a displayed on the operation panel 120. Note that, as will be described later, many icons (also referred to as operation keys) are arranged on the actual operation screen, but in FIG. A plurality of icons i11 to i14 (hereinafter collectively referred to as “icon i” (the same applies to the icons described later)) are arranged and visually displayed. FIG. 6 shows an operation screen 22a (Braille image) for Braille displaying Braille image data d1 to d4 for Braille (hereinafter collectively referred to as “Braille image data d”) corresponding to FIG. The Braille information included in the Braille image data d1 to d4 corresponds to the character data indicated by the icons i11 to i14 on a one-to-one basis. The image data for the operation screen described later and FIG. 5 and the braille data (image data) for Braille as shown in FIG. 6 are both stored in the storage unit 130.

  In the normal mode, the process corresponding to each icon i is executed by touching each icon i with a finger. For example, when the icon i11 is touched, the operation screen changes to a setting screen related to copying. In the visually handicapped person mode to be described later, when the user wearing the braille device 200 touches the icon i on the operation screen with the finger cover 292, the corresponding braille image data d is displayed on the braille display unit 220. Specifically, for example, by touching any position in the frame (square frame) of the icon i11 on the surface of the operation panel 120 with the finger cover 292, the Braille image data d1 corresponding to the icon i11 is obtained from the main body device 100. It is transmitted to the braille device 200 by wireless communication. Then, the braille device 200 displays the received braille image data d1 on the braille display unit 220. This control will be described later. In the example of FIG. 6, the braille image data d1 is composed of data corresponding to 6 pins, with 6 pins as a set of data. Each data may be sequentially displayed by the braille display unit 220 at predetermined time intervals. Alternatively, six pieces of data may be displayed while being scrolled by a slide operation in the icon i11 by the user. Further, the braille image data d1 may include not only the braille information but also a figure such as a frame surrounding the whole.

(Decision switch 230)
The decision switch 230 is a pressure sensor arranged so as to overlap the braille display unit 220. When the user presses the finger F downward and a pressure higher than a predetermined level is detected by the pressure sensor, the control unit 210 determines that the determination switch 230 has been turned ON by the user. The determination switch 230 is not limited to this, and may be a tact switch operated with a free hand. In this case, the tact switch may be disposed on the upper part of the unit case 293.

(Contact sensor 240)
The contact sensor 240 functions as a “sensor unit”. The contact sensor 240 is disposed at a position corresponding to the belly of the finger of the finger cover 292 and detects that the outside of the finger cover 292 has contacted an object such as the surface of the operation panel 120. The contact sensor 240 may be a mechanical sensor such as a micro switch or a pressure sensor, or may be an optical sensor. Further, when a pressure sensor is used, it may be determined that a contact has been made when an object is contacted at a pressure higher than a predetermined pressure. When the user wearing the braille device 200 touches any object such as the operation panel 120 with the tip of the finger cover 292, the contact sensor 240 detects the touch.

(Control method)
Hereinafter, a control method executed by the control system 500 will be described with reference to FIGS. 7 to 8C. FIG. 7 is a flowchart showing the control on the Braille device 200 side. FIG. 8A is a main flowchart showing control on the main device 100 side, and FIGS. 8B and 8C are flowcharts of subroutines.

(Control on the braille device 200 side)
FIG. 7 is a flowchart showing the control on the Braille device 200 side. The control of FIG. 7 is performed in response to the establishment of wireless communication with the main body device 100 in a procedure in accordance with a predetermined communication standard.

(Step S11)
First, the control unit 210 of the braille device 200 determines whether or not the mode is shifted to the visually impaired person mode after communication with the main body device 100 is established. If the wireless communication by the BLE, for example, BLE can be established with the braille device 200, the main body device 100 determines the shift to the visually impaired person mode and transmits the determination result to the braille device 200 side. The braille device 200 waits until it receives a result indicating that it has shifted to the visually handicapped person mode (NO), and if it receives the shift result (YES), the process proceeds to step S12.

(Step S12)
Here, control unit 210 determines whether or not Braille information has been received from main device 100. If braille information has been received (YES), the process proceeds to step S13.

(Step S13)
A braille image corresponding to the received braille information is displayed on the braille display unit 220. As will be described later, the user can recognize the content of the icon i if the information is Braille information corresponding to the touch on the icon i.

(Step S14)
The user recognizes the content of the icon i from the display content of the braille display unit 220. If it is the desired icon i, the determination switch 230 is turned on (pressed down). When the determination switch 230 is turned on (YES), the controller 210 advances the process to step S15. If it is not ON (NO), the process is terminated, and the processes after step S11 are repeated.

(Step S15)
The braille device 200 transmits an ON signal of the determination switch 230 to the main body device 100 and ends.

(Control on the main device 100 side)
(Step S21)
The control unit 110 of the main device 100 determines whether or not communication with the braille device 200 is possible. If wireless communication with the braille device 200 cannot be established (NO), the process proceeds to step S22 in which switching to the normal mode permitting the drag operation is performed. If communication can be established (YES), the process proceeds to step S23 for switching to the visually impaired person mode in which the drag operation is prohibited. It should be noted that switching to the audiovisual disabled person mode is performed after communication between the main body device 100 and the braille device 200 is established, and further after receiving a confirmation instruction from the user by pressing the decision switch 230 by the braille device 200 or the like. It may be. Further, the determination in step S21 may be sequentially performed at a predetermined cycle, and the visually impaired person mode may be switched to the normal mode in response to the established communication being disconnected.

(Step S22 (subroutine flowchart of “normal mode”))
FIG. 8B is a subroutine flowchart relating to the normal mode of step S22.

  Here, various operations of the touch panel 122 will be described. In the following, “touch” means a case where touch panel 122 is touched by a user's finger (may be a body part other than the finger or an object such as Braille device 200 possessed by the user). This touch includes the following drag operations and click operations. The drag operation is an operation of sliding the finger in a predetermined direction with the finger lightly touching the touch panel 122, and is also referred to as a swipe operation or “tracing”. “Trace” is different from drag operation and swipe operation in that it includes an operation of changing the moving direction while sliding a finger. The click operation is an operation of releasing (separating) a finger touching one point of the touch panel 122 without changing the touch position within a predetermined time.

(Step S31)
Here, control unit 110 determines whether a touch on operation panel 120 has been detected. If a touch is detected, the process proceeds to step S32.

(Step S32)
The control unit 110 determines whether or not the detected touch is a drag operation in the scroll area. If it is determined that the operation is a drag operation to the scroll area (YES), the process proceeds to step S33; otherwise, the process proceeds to step S34.

(Step S33)
Here, the icons in the scroll area are scrolled and displayed. Here, the “scroll area” and the scroll display performed in step S33 will be described with reference to the drawings. FIG. 9A is an example of an operation screen displayed on the operation panel in the normal mode, and FIG. 9B is a diagram illustrating the operation screen after the drag operation in FIG. 9A.

  As shown in FIG. 9A, a scroll area a1 is provided at the bottom of the operation screen 12b. The scroll area a1 is an area where scroll display is performed. If a drag operation is performed within the scroll area a1, it is determined to be valid and the icon i is scrolled. Here, in the operation screen 12b shown below, for example, a total of eight icons i1 to i8 are arranged in the scroll direction (left-right direction) in the scroll area a1, and one of the eight icons is shown in the figure. Icons i1 to i5 are displayed. The user can scroll the icon i in the direction in which the finger is slid by performing a drag operation toward the left or right direction in the scroll area a1. FIG. 9B shows a state in which three icons i in the scroll area a1 are scrolled to the right by three with respect to FIG. 9A in response to a drag operation of sliding a finger in the right direction.

(Step S34)
Here, the control unit 110 determines whether the detected touch is a touch on any icon i. If the touch is on the icon i (YES), the process proceeds to step S35; otherwise, the process returns to step S31, and the subsequent processes are repeated.

(Step S35)
Since the selection operation for the icon i has been performed, processing corresponding to the selected icon i is executed. For example, when an operation for selecting the icon i4 is performed, the operation screen is displayed for selecting a paper size.

(Step S23 (subroutine flowchart of “visually impaired person mode”))
FIG. 8C is a subroutine flowchart regarding the visually impaired person mode in step S23. When the mode is shifted to the visually impaired person mode, first, or periodically, information indicating that the visually impaired person mode is being executed may be transmitted to the braille device 200 (FIG. 7). (See step S11).

(Step S41)
Control unit 110 determines whether a touch on operation panel 120 has been detected. If a touch is detected, the process proceeds to step S42.

(Step S42)
The control unit 110 determines whether or not the detected touch is a drag operation in the scroll area. If it is determined that the drag operation is in the scroll area (YES), the process proceeds to step S43, and if not, the process proceeds to step S44.

(Step S43)
Here, the icon i in the scroll area is not moved. Here, the process performed in step S43 will be described with reference to the drawings. FIG. 10A is an example of an operation screen displayed on the operation panel in the visually impaired person mode. As shown in FIG. 10A, FIG. 10A is an operation screen common to FIG. 9A. That is, in this embodiment, even if the normal mode and the visually impaired person mode are switched between each other, the content displayed on the operation panel 120 is not changed and remains as it is. By making the display content of the operation screen common in the normal mode and the visually impaired person mode, it is not necessary to prepare a new operation screen for the visually impaired person mode, so that the man-hours related to the creation of display data can be reduced.

  FIG. 10B is a diagram showing an operation screen after the drag operation in FIG. 10A. As shown in FIG. 10B, in the visually impaired mode, the icon i in the scroll area a1 is not moved even if a drag operation is performed on the scroll area a1. In FIG. 8C, the processing of steps S42 and S43 indicated by the broken line does not change at all, and thus the processing may be omitted (the same applies to the processing of the broken line in FIG. 11 described later).

(Step S44)
Here, the control unit 110 determines whether the detected touch is a touch on any icon i. If the touch is on the icon (YES), the process proceeds to step S45; otherwise, the process returns to step S41, and the subsequent processes are repeated.

(Step S45)
Braille information is transmitted to the Braille device 200 according to the content of the icon. For example, as shown in FIG. 10B, if the touch is inside the frame of the icon i2, Braille information corresponding to the icon i2 is transmitted to the Braille device 200. When the user slides the touch position to the right from the icons i5 to i2 while touching the operation panel 120 with the finger between FIG. 10A and FIG. 10B, the control unit 110 displays the icon i4. , I3 and i2 are sequentially transmitted to the braille device 200 worn by the user.

(Step S46)
Here, it is determined whether or not the ON signal of the determination switch 230 from the Braille device 200 has been received in accordance with the process of Step S15 of the Braille device 200 of FIG. If an ON signal has been received (YES), the process proceeds to step S47. If not (NO), the process returns to step S41, and the subsequent processes are repeated.

(Step S47)
Here, similarly to step S <b> 35, the control unit 110 treats reception of the ON signal of the determination switch as a selection operation, and executes a function process corresponding to the icon i currently touched by the braille device 200.

  As described above, in the control system 500 using the Braille device 200 according to the present embodiment, the control unit 110 displays the scroll area when the scroll area displayed in the normal mode is displayed on the operation screen in the visually impaired mode. Control is performed so that the icon i in the scroll area is not moved even when a drag operation is performed on the icon. As a result, when a visually handicapped person operates an operation screen with a scroll area, the icon i in the scroll area moves unintentionally, and the problem that the icon i in the scroll area cannot be recognized correctly is prevented. be able to. In addition, the operation screen used by the healthy person can be shared without the visually handicapped person being confused by the operation.

(Second Embodiment)
Next, a second embodiment will be described. In the second embodiment, it is determined whether or not the touch (drag operation) performed during the execution of the visually impaired person mode is performed by the Braille device 200, and the control is switched according to the result.

  FIG. 11 is a subroutine flowchart showing the control on the main device side according to the second embodiment. Since the flowchart on the main body side other than that shown in the figure, that is, the processes in FIGS. 8A and 8B are the same as those in the first embodiment, description thereof will be omitted. A part of the processing common to FIG.

(Steps S41 and S42)
If the same processing as that in FIG. 8C is performed and a drag operation to the scroll area a1 is performed in the visually impaired person mode, the processing proceeds to step S421.

(Step S421)
Here, control unit 110 determines whether or not the operation is a drag operation by braille device 200. If it is a drag operation by the braille device 200 (YES), the process proceeds to step S43; otherwise (NO), and if it is a drag operation by other than the Braille device 200 (NO), the process proceeds to step S422. The determination as to whether or not it is based on the braille device 200 is performed as follows.

  (A) The Braille device 200 monitors the output of the contact sensor 240, and transmits sensor information, that is, information on contact timing to any object to the main body device 100 in real time. It is preferable to transmit the sensor information at a predetermined cycle, for example, a cycle of several tens of Hz. (B) The control unit 110 of the main device 100 detects a touch on the operation panel 120 of the own device. At this time, it is performed depending on whether or not the waveform indicating the contact timing of the sensor information (a) transmitted from the Braille device 200 is the same as the waveform indicating the timing of touching the operation panel 120. More specifically, if the timing at which the sensor information changes from non-contact to contact and the signal waveform indicating the continuation of the subsequent touch match the signal waveform detected by the touch in the main device 100 in step S41, It is determined that the touch is made by the braille device 200. For example, when sensor information is received from the braille device 200 at a cycle of 60 Hz, the main body device 100 determines that the timing at which the main device 100 changes from non-contact to contact is the same cycle or a difference of about one cycle (about 16 msec). It is determined that this is a touch by 200 and a drag operation associated therewith.

(Step S422)
The scroll area icon i is scroll-displayed in response to a drag operation performed with a finger other than the finger wearing the braille device 200, for example, a finger of a healthy person or a finger other than the index finger of the right hand. This process is the same as the normal mode, which is the same as step S33 in FIG. 8B.

(Step S43)
Similar to the processing described with reference to FIG. 8C, the icon in the scroll area is not moved here.

(Steps S44 to S47)
This is the same process as in FIG.

  As described above, in the second embodiment, when the drag operation on the scroll area is not a drag operation by the Braille device, the icon is moved. In this way, the same effects as those of the first embodiment can be obtained. For example, in a visually impaired person mode, when a healthy person uses an operation panel together with a visually impaired person, a special operation is performed. It can be shared by visually impaired and healthy people without need.

(Third embodiment)
Next, a third embodiment will be described. In the first embodiment, the icon in the scroll area a1 cannot be scrolled during the visually impaired person mode. In the third embodiment, an icon in the scroll area a1 is shifted using a scroll button.

  FIG. 12 is a subroutine flowchart showing control on the main device side according to the third embodiment. FIG. 8 is a process performed subsequent to step S41 (YES) in FIG. 8C. Processes other than the process shown in FIG. 8 are the same as those in the first embodiment, and a description thereof is omitted. 13A and 13B are examples of the operation screen 12b displayed on the operation panel in the visually impaired person mode, and these operation screens 12b are operation screens common to FIG. 9A and the like.

(Step S51)
The control unit 110 determines whether or not a selection operation has been performed on the right scroll button b1. Specifically, as a selection operation, when the ON signal of the determination switch 230 is received from the braille device 200 while the right scroll button b1 is being touched in step S41 (YES), the process proceeds to step S55. If not (NO), the process proceeds to step S52.

(Step S52)
Next, the control part 110 determines whether selection operation to the left scroll button b2 was performed by the procedure similar to step S51. If it is a selection operation to left scroll button b2 (YES), the process proceeds to step S53, and if not (NO), the process proceeds to step S57.

(Step S53)
The control unit 110 determines whether or not there is an icon i hidden on the left side of the icon i displayed in the scroll area a1 in the current display state. If there is a hidden icon i (YES), the process proceeds to step S54, and if there is no icon i, the process proceeds to step S57.

(Step S54)
The control unit 110 slides (moves) one icon i in the scroll area a1 to the right. The operation screen 12b in FIG. 13B is a diagram showing a state after the user selects and operates the left scroll button b2 three times with respect to the operation screen 12b in FIG. 13A. In FIG. 13A, since there are icons i6 to i8 hidden behind the icon i5, in FIG. 13B, the scroll area is displayed so that the three icons i hidden on the left are displayed according to the number of selection operations. The icon i in a1 is slid three times to the right.

(Step S55)
Here, the control unit 110 determines whether or not there is an icon i hidden on the right side of the icon i displayed in the scroll area a1. If there is a hidden icon i (YES), the process proceeds to step S56, and if there is no icon i, the process proceeds to step S57.

(Step S56)
The control unit 110 slides one icon i in the scroll area a1 to the left.

(Step S57)
The controller 110 does not move the icon i in the scroll area a1, and maintains the display state as it is.

  As described above, in the third embodiment, the same effect as that of the first embodiment can be obtained, and further, by using the scroll button to shift the icon in the scroll area a1, the visual impairment can be achieved. In the user mode, the hidden icon i in the scroll area can be displayed.

(First modification)
FIG. 14 is a conceptual diagram showing an operation screen 22b (Braille image) for Braille, and corresponds to the operation screen 12b (viewing image) of FIG. 13B. In the state of FIG. 13B, icons i8 to i4 are displayed in the scroll area a1. Since the icon i8 is located at the left end, there is no hidden icon i on the left side of the icon i8. The control unit 110 does not display the scroll button b1 (or b2) on the Braille operation screen (Braille image) according to this state (Braille display). Specifically, the control unit 110 does not transmit Braille information to the Braille device 200 even when the position of the scroll button b1 (or b2) in FIG. 13B is touched. By doing so, the visually impaired person can recognize that there is no other icon i beyond the icon i8 in the scroll area a1.

(Fourth embodiment)
In the third embodiment, the scroll buttons b1 and b2 are used to slide the icon i in the scroll area a1. In the fourth embodiment, the scroll buttons b1 and b2 are further provided with different functions.

  FIG. 15 is a diagram illustrating the functions of the scroll buttons b1 and b2 on the operation screen 12b. The control unit 110 detects the touch to the position of the left scroll button b2 in FIG. 15 and transmits the braille information associated with the left scroll button b2 to the braille device 200. Specifically, since the two icons i are hidden behind the leftmost icon i6 currently displayed in the scroll area a1, Braille data representing “2” is transmitted as Braille information. It should be noted that the number of icons i currently displayed (5) in place of or in addition to the number of icons i hidden behind the touch position, that is, in the direction of the left scroll button b2 (left direction), Alternatively, the total number of icons (eight) may be transmitted to the braille device 200. Further, if there is no icon i hidden behind the touch, information indicating that there is no icon i may be transmitted to the Braille device 200. Thus, the visually impaired can correctly recognize the display state of the scroll area a1.

(Second modification)
FIG. 16 is a diagram for explaining a function of a boundary portion between icons i adjacent to each other in the scroll area a1 in the second modified example. As shown in the figure, in the second modification, when a touch operation is performed on the boundary portions a21, a22, a23, and a24 between the icons i displayed in the scroll area a1, information on the arrangement state of the icons i is displayed. Is notified to the braille device 200 as braille information.

  Specifically, as illustrated in FIG. 16, when the user touches the boundary portion a <b> 22 while sliding the finger from the left (drag operation), the control unit 110 moves 2 to the right with respect to the Braille device 200. Braille data of “Migiko 2” indicating that the individual icon i is displayed is transmitted to the Braille device 200. Note that the arrangement state is not limited to this, braille information indicating that there are three on the left side may be notified to the braille device 200, and braille information indicating that there are two on the right side and three on the left side may be notified. Also good. Also, the braille information indicating the serial number of the icon i immediately ahead (right immediately when the finger is slid from left to right) may be notified.

(Third Modification)
In the present embodiment, as described above, as a method for notifying Braille information, the display content of the Braille display unit 220 is displayed so as to be shifted by the amount of shift of the touch position on the touch panel 122. Specifically, for example, when the touch position is shifted 1 mm to the right, the display content of the braille display unit 220 (the protruding position of the pin) is displayed to be shifted 1 mm to the right. When displaying in this way, if the area of the boundary portions a21 to a24 is narrow, the number of characters that can be displayed in Braille is limited, and there is a possibility that sufficient information cannot be notified. For such a problem, in the third modified example, a Braille image may be displayed in an empty portion where there is no other icon i around the boundary portion (left, right, up, down, diagonal).

  FIG. 17 is a diagram illustrating the function of the boundary portion between icons i in the third modification. In the example of FIG. 17, in the boundary portion a <b> 22, a braille image related to a graphic indicating “up” or an upward arrow for guiding the touch position to a predetermined area is displayed. Information regarding the arrangement state of the icon i as in the second modification is displayed in the empty area a32 in the direction indicating the braille image and having no display content. By doing in this way, the user who touched the boundary part a22 is guide | induced upwards by reading a braille image. The user can recognize the information regarding the arrangement state by sliding the touch position upward. By doing in this way, even if it is a case where the boundary parts a21-a24 between the icons i are narrow and a sufficient number of characters cannot be displayed as braille data, it is sufficient to display them in the corresponding empty areas a31-a34. Information can be displayed on the braille device 200.

(Fifth embodiment)
FIG. 18 is a flowchart showing control on the main device 100 side according to the fifth embodiment. The flowchart of FIG. 8 corresponds to the flowchart of FIG. 8A, and steps S61, S62, and S64 correspond to steps S21, S23, and S22 of FIG. 8A, respectively.

(Steps S61 and S62)
Control unit 110 proceeds to step S63 after switching to the visually impaired person mode in response to communication with braille device 200 being enabled.

(Step S63)
The control unit 110 continues to determine whether or not a time-out due to no operation by the braille device 200 has occurred. If a timeout has occurred (YES), the process proceeds to step S64. Specifically, when the touch operation on the operation panel 120 is not continuously performed for a predetermined time or longer (for example, several tens of seconds or several minutes), it is determined that a timeout has occurred.

(Step S64)
The control unit 110 switches from the visually impaired person mode to the normal mode, and enables the drag operation to the scroll area a1.

  By doing so, it is possible to prevent the visually impaired person mode from being inadvertently continued.

(Sixth embodiment)
FIG. 19 is a flowchart showing control on the main device 100 side according to the sixth embodiment. The flowchart of FIG. 8 corresponds to the flowchart of FIG. 8A, and steps S71, S73, and S74 correspond to steps S21, S22, and S23 of FIG. 8A, respectively.

(Step S71)
If it is not possible to communicate with Braille device 200 (NO), control unit 110 advances the process to step S73. If communication is possible (YES), the process proceeds to step S72.

(Step S72)
Control unit 110 waits for an input of a switching instruction (NO). If an instruction to switch to the normal mode is input, the process proceeds to step S73. If an instruction to switch to the visually impaired person mode is input, the process proceeds to step S74.

  This switching instruction may be performed, for example, by operating a selection button (not shown) of the main body device 100 provided around the operation panel 120, or the determination switch 230 may be used as a switching instruction button. Good. In the latter case, for example, if the decision switch 230 is pressed during execution of the visually impaired person mode, the mode is switched to the normal mode, and if the same determination switch 230 is pressed during execution of the normal mode, the mode is changed to the visually impaired person mode. Switch.

(Steps S73, S74)
Similar to steps S23 and S22 of FIG. 8A, the control unit 110 switches to the normal mode in step S73, and switches to the visually impaired person mode in step S74.

  By doing so, the visually impaired person mode and the normal mode can be switched between each other at an arbitrary timing, and the visually impaired person and the healthy person can share the operation of the operation panel 120.

  The configuration of the control system 500 including the main body device 100 and the braille device 200 described above is the main configuration in describing the features of the above-described embodiment, and is not limited to the above configuration. Within the scope, various modifications can be made as described below. Further, the configuration of the general main device 100 and the braille device 200 is not excluded.

  In the above-described embodiment, an example in which the braille display unit 220 of the braille device 200 is used as the notification unit has been described, but instead of this, the speaker 180 may be used as the notification unit. Specifically, the display content of the operation screen displayed in each embodiment is notified to a visually impaired person through a speaker by voice.

  The means and method for performing various processes in the control system according to each of the above-described embodiments and modifications can be realized by either a dedicated hardware circuit or a programmed computer. The program may be provided by a computer-readable recording medium such as a CD-ROM (Compact Disc Read Only Memory), or may be provided online via a network such as the Internet. In this case, the program recorded on the computer-readable recording medium is usually transferred to and stored in a storage unit such as a hard disk. The program may be provided as a single application software, or may be incorporated into the software of the apparatus as one function of the Braille device cooperation system.

500 Braille Device Linkage System 100 Main Body Device 110 Control Unit 120 Touch Panel 170 Wireless Communication Unit 180 Speaker 200 Braille Device 210 Control Unit 220 Contact Sensor 230 Braille Display Unit 240 Determination Switch 250 Wireless Communication Unit 291 Glove Main Body 292 Finger Cover 293 Unit Case

Claims (13)

A touch panel for displaying an operation screen on the display surface and detecting a touch on the display surface by a user;
A notification unit for notifying the visually impaired,
A display control unit that controls display contents of the operation screen, and executes a process associated with the icon in response to a selection operation on any of the plurality of icons displayed on the operation screen;
Notifying the user of the display content of the operation screen at the touch position on the touch panel via the notification unit, and performing the selection operation of the icon at the touch position by operating a determination switch different from the touch panel. A mode switching unit that switches between a visually impaired person mode and a normal mode for performing the selection operation of the icon by touching the icon on the touch panel;
The display control unit
In the normal mode, a scroll area is provided in the operation screen, a plurality of the icons are arranged in a predetermined scroll direction in the scroll area, and displayed in a scrollable manner by a drag operation by a user.
In the visually impaired person mode, when the scroll area displayed in the normal mode is displayed on the operation screen, the icon in the scroll area is not moved even if a drag operation is performed on the scroll area. Control system. It has a braille device worn by the user,
The braille device is
A finger cover covering the user's fingertips;
The notification unit arranged inside the finger cover and capable of notifying braille information through touch to a visually impaired person;
The decision switch;
A communication unit that wirelessly communicates with the main body device provided with the touch panel,
The control system according to claim 1, wherein the display control unit notifies the user of braille information related to the display content of the operation screen at a touch position on the touch panel by the notification unit.   3. The control according to claim 2, wherein, even in the visually impaired mode, when the drag operation is performed on the scroll area, the icon is moved when the drag operation is not performed by the Braille device. system. The braille device is further provided with a sensor unit that is disposed at a position corresponding to the belly of the finger of the finger cover and detects that the outside of the finger cover has contacted an object,
The display control unit determines whether the drag operation on the scroll area is performed by the Braille device based on a contact timing detected by the sensor unit transmitted from the Braille device and a touch timing detected by the touch panel. The control system according to claim 3, wherein it is determined whether or not. On the operation screen, a button for moving the icon is arranged in an area other than the scroll area,
The control system according to any one of claims 1 to 4, wherein the display control unit moves the icon in the scroll region in response to a selection operation on the button in the visually impaired person mode. The button is arranged at a position adjacent to the scroll area in the scroll direction,
The display control unit, in the visually impaired mode, in response to a touch operation on the button, the number of the icons that are hidden in the direction of the button, the number of the icons that are displayed, and display and hidden The control system according to claim 5, wherein the notification unit notifies a total number of the icons, which is a total of the icons. The display control unit, in the visually impaired person mode,
When a touch operation or a drag operation is performed on a boundary portion between two adjacent icons in the scroll area, the notification unit notifies information related to an arrangement state of the icons in the scroll area. The control system according to any one of claims 1 to 6.   5. The mode switching unit according to claim 2, wherein the mode switching unit switches to the visually impaired person mode in response to establishment of wireless communication between the main body device and the Braille device. Control system.   9. The mode switching unit according to claim 8, wherein the mode switching unit switches from the visually impaired mode to the normal mode in response to disconnection of wireless communication established between the main body device and the braille device. Control system.   When the mode switching unit switches to the visually impaired mode in response to the establishment of wireless communication between the main body device and the braille device, the mode switching unit switches to the last, or finally, the braille device. 10. The control system according to claim 8, wherein the operation mode is switched to the normal mode when a state in which the operation by the braille device is not performed for a predetermined time or longer after the operation is performed.   When the mode switching unit switches to the visually impaired mode in response to the establishment of wireless communication between the main body device and the braille device, the mode switching unit is configured to The control system according to any one of claims 8 to 10, wherein the control system is switched to a normal mode or the visually impaired person mode.   The control system according to claim 1, wherein a common operation screen is displayed in the normal mode and the visually impaired person mode. A control program that is executed by a computer that controls a control system that displays an operation screen on a display surface, and that includes a touch panel that detects a touch on the display surface by a user and a notification unit that notifies a visually impaired person of information. Because
A step (a) of controlling the display content of the operation screen and executing a process associated with the icon in response to a selection operation on any of the plurality of icons displayed on the operation screen;
Visual information for notifying the user of the display content of the operation screen at the touch position on the touch panel via the notification unit and performing the selection operation of the icon at the touch position by operating a determination switch different from the touch panel. (B) switching between a disabled person mode and a normal mode for performing the selection operation of the icon by touching the icon on the touch panel,
In the step (b), in the normal mode, a scroll area is provided in the operation screen, a plurality of the icons are arranged in a predetermined scroll direction in the scroll area, and scrollable by a drag operation by a user. Display
In the visually impaired person mode, when the scroll area displayed in the normal mode is displayed on the operation screen, the icon in the scroll area is not moved even if a drag operation is performed on the scroll area. A control program for causing the computer to execute processing.