WO2020001421A1 - 语音屏幕鼠标校核下的图标安全操作方法 - Google Patents

语音屏幕鼠标校核下的图标安全操作方法 Download PDF

Info

Publication number
WO2020001421A1
WO2020001421A1 PCT/CN2019/092714 CN2019092714W WO2020001421A1 WO 2020001421 A1 WO2020001421 A1 WO 2020001421A1 CN 2019092714 W CN2019092714 W CN 2019092714W WO 2020001421 A1 WO2020001421 A1 WO 2020001421A1
Authority
WO
WIPO (PCT)
Prior art keywords
icon
voice
screen
mouse
check
Prior art date
Application number
PCT/CN2019/092714
Other languages
English (en)
French (fr)
Inventor
王立松
Original Assignee
南京铁道职业技术学院
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 南京铁道职业技术学院 filed Critical 南京铁道职业技术学院
Priority to US17/256,650 priority Critical patent/US11656738B2/en
Publication of WO2020001421A1 publication Critical patent/WO2020001421A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/03—Arrangements for converting the position or the displacement of a member into a coded form
    • G06F3/033—Pointing devices displaced or positioned by the user, e.g. mice, trackballs, pens or joysticks; Accessories therefor
    • G06F3/0354—Pointing devices displaced or positioned by the user, e.g. mice, trackballs, pens or joysticks; Accessories therefor with detection of 2D relative movements between the device, or an operating part thereof, and a plane or surface, e.g. 2D mice, trackballs, pens or pucks
    • G06F3/03545—Pens or stylus
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0481—Interaction techniques based on graphical user interfaces [GUI] based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
    • G06F3/04817—Interaction techniques based on graphical user interfaces [GUI] based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance using icons
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B61—RAILWAYS
    • B61L—GUIDING RAILWAY TRAFFIC; ENSURING THE SAFETY OF RAILWAY TRAFFIC
    • B61L25/00—Recording or indicating positions or identities of vehicles or trains or setting of track apparatus
    • B61L25/06—Indicating or recording the setting of track apparatus, e.g. of points, of signals
    • B61L25/08—Diagrammatic displays
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/03—Arrangements for converting the position or the displacement of a member into a coded form
    • G06F3/033—Pointing devices displaced or positioned by the user, e.g. mice, trackballs, pens or joysticks; Accessories therefor
    • G06F3/0354—Pointing devices displaced or positioned by the user, e.g. mice, trackballs, pens or joysticks; Accessories therefor with detection of 2D relative movements between the device, or an operating part thereof, and a plane or surface, e.g. 2D mice, trackballs, pens or pucks
    • G06F3/03543—Mice or pucks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/03—Arrangements for converting the position or the displacement of a member into a coded form
    • G06F3/033—Pointing devices displaced or positioned by the user, e.g. mice, trackballs, pens or joysticks; Accessories therefor
    • G06F3/038—Control and interface arrangements therefor, e.g. drivers or device-embedded control circuitry
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0481—Interaction techniques based on graphical user interfaces [GUI] based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
    • G06F3/04812—Interaction techniques based on cursor appearance or behaviour, e.g. being affected by the presence of displayed objects
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0484—Interaction techniques based on graphical user interfaces [GUI] for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0487—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser
    • G06F3/0488—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser using a touch-screen or digitiser, e.g. input of commands through traced gestures
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/16—Sound input; Sound output
    • G06F3/167—Audio in a user interface, e.g. using voice commands for navigating, audio feedback
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L15/00—Speech recognition
    • G10L15/22—Procedures used during a speech recognition process, e.g. man-machine dialogue
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B61—RAILWAYS
    • B61L—GUIDING RAILWAY TRAFFIC; ENSURING THE SAFETY OF RAILWAY TRAFFIC
    • B61L27/00—Central railway traffic control systems; Trackside control; Communication systems specially adapted therefor
    • B61L27/10—Operations, e.g. scheduling or time tables
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2203/00—Indexing scheme relating to G06F3/00 - G06F3/048
    • G06F2203/038—Indexing scheme relating to G06F3/038
    • G06F2203/0381—Multimodal input, i.e. interface arrangements enabling the user to issue commands by simultaneous use of input devices of different nature, e.g. voice plus gesture on digitizer
    • G—PHYSICS
    • G08—SIGNALLING
    • G08B—SIGNALLING OR CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS
    • G08B21/00—Alarms responsive to a single specified undesired or abnormal condition and not otherwise provided for
    • G08B21/18—Status alarms
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L15/00—Speech recognition
    • G10L15/22—Procedures used during a speech recognition process, e.g. man-machine dialogue
    • G10L2015/223—Execution procedure of a spoken command

Definitions

  • the invention belongs to the field of security technology, and particularly relates to a method for safely operating an icon under a voice screen mouse check.
  • misoperations There are two types of misoperations: one is that the operation performed is consistent with the subjective wishes of the operator at that time, but objectively caused adverse consequences; the second is that the operation performed is inconsistent with the subjective wishes of the operator at that time, but Caused by accidental hit on the screen.
  • the first type of misoperation has nothing to do with the equipment and cannot be prevented on the technical equipment, it is completely determined by the technical level of the person.
  • the second type of misoperation is an error in the process of human-computer interaction, which is caused by the careless operation of some people and the inconvenience of device operation.
  • the invention is directed to the second misoperation.
  • the first is to strengthen the training of operators, improve the professional quality of the operators, strict rules and regulations, and strive to ensure that the operators carefully confirm before operation. This approach is a management issue.
  • the second is to increase the redundancy of the system when designing the system, and to ensure that the operation is correct by reconfirming the prompt or double- or even multi-person operation check. This approach is technical.
  • the present invention provides a method for safe operation of icons under voice screen mouse check to reduce errors in human-computer interaction.
  • the so-called ternary check of voice, screen and mouse refers to the "voice icon” associated with the operator's spoken content, the "screen icon” displayed on the screen to specify the operation, and the "mouse icon” pointed to when the mouse stops moving. Exactly, otherwise the system will issue an alarm or prohibit operation.
  • the present invention adopts the following technical solutions:
  • a method for safe operation of an icon under a ternary check of a voice screen mouse which is characterized in that an icon associated with the operator's voice, an icon for specifying an operation displayed on the screen, and an icon pointed to when the mouse stops moving are calibrated.
  • Check and compare the system gives the operation authority and warning; first establish the association between the specific voice and the icon, activate the associated icon through the voice input of the voice recognition system, and then check and compare with the icon pointed when the mouse stops moving .
  • Step 1 Set the association between the voice and the screen icon: set the correspondence between the voice and the screen icon, and construct the voice icon;
  • Step 2 Associate the voice recognition system: When the system receives a specific voice, it will activate the corresponding screen icon;
  • Step 3 Set the association between the mouse and the screen icon: When the mouse stays in the icon area that is not activated by the voice, set the association between the mouse and the screen icon according to the level of safety production requirements;
  • Step 4 Set the screen area when the mouse is not operated: Set the parking area when the mouse is not operated on the screen.
  • the first step there is a one-to-one correspondence between voice and screen icons, many-to-one, or one-to-many.
  • one or more screen icons can be activated by a specific voice according to the setting of the association information between the voice and the screen icons.
  • the association setting of the mouse and the screen icon is set to two levels according to the level requirements of safety production: the first level, the mouse is activated and an alarm is issued, and the operation is prohibited; the second level, the mouse is activated and an alarm is issued, but it can be Operation; when the mouse is hovering over the voice-activated icon area, it can operate normally.
  • the speech recognition system automatically turns off the voice receiving function; when the mouse is not operated, if the mouse is not placed in the parking area for more than a predetermined set time, an alarm will be displayed in the area.
  • a method for safe operation of icons under the ternary check of the voice screen mouse is also proposed, which is characterized in that a shadow system is provided for the protection of the existing system, and the shadow system is provided on the original system operation screen with the size of the original system.
  • Identical and one-to-one shadow icons Use the shadow icons instead of the corresponding original system icons to establish the associated information of voice and shadow icons.
  • the shadow system operates independently, and has no effect on the operation of the original system, and is only used for voice. , Icon and mouse operation check.
  • a method for safe operation of icons under the dual check of voice screen is also proposed, which is characterized in that for a touch screen computer or a touch screen mobile terminal, an icon associated with a voice issued by an operator is displayed on the screen to specify an operation Icon for binary check, the system gives the operation authority.
  • a method for safe operation of icons under the dual check of the voice screen is also proposed, which is characterized in that a shadow system is provided for a touch screen computer or a touch screen mobile terminal to protect the existing system, and the shadow system operates the screen on the original system
  • shadow icons that are exactly the same size as the original system and have a one-to-one correspondence.
  • the shadow icon is used to replace the corresponding original system icon to establish the associated information of the voice and shadow icons.
  • the shadow system operates independently and operates on the original system No effect, only for voice and icon operation check.
  • the present invention provides a method for ternary checking of voice, screen and mouse for safe operation to reduce errors in human-computer interaction. Especially in the field of high safety, it is required to strictly and carefully confirm before operation, so that the "eyes, hands, and mouths" are consistent, but this all depends heavily on the quality of people.
  • This method uses modern speech recognition tools to check human behavior through a computer system to ensure correct operation to the greatest extent and avoid accidents.
  • the present invention proposes the concept and method of "shadow system". Since the shadow system is independently developed and runs independently, it is only used for voice, screen and mouse operation verification, and has no impact on the operation of the original system. This makes it possible for the existing system to provide independent security operation protection software, which is easy to develop and more practical.
  • FIG. 1 is a system working flowchart of the next icon operation of the ternary check.
  • FIG. 2 is a system work flow chart of the next icon operation of the binary check.
  • Figure 3 is a schematic diagram of a double-line automatic blocking railway station console.
  • association information of the voice and the screen icon is to set the correspondence between the voice and the screen icon, and construct a voice icon.
  • the relationship between speech and icons is a one-to-one correspondence in principle, but when the system is more complex, emphasizing the one-to-one correspondence may make the speech too complicated and cause inconvenience to the operation. Therefore, it can be designed into many-to-one and one-to-many according to the actual application, voice complexity and security level requirements.
  • Associated speech recognition system means that when the system receives a specific voice, it will activate the corresponding screen icon. According to the setting of the information related to the voice and the screen icon, a specific voice can activate one screen icon or multiple screen icons.
  • the association setting of the mouse and the icon can be set to the second level according to the level requirements of safety production.
  • the first level the mouse is activated and an alarm is issued, and it cannot be operated. This is the safest setting.
  • the second level is an operational setting, that is, the mouse is activated and alarmed, but it can be operated. At this time, the operator is reminded to strengthen the confirmation.
  • the mouse hovering over the voice-activated icon area, it can operate normally.
  • the operating system workflow of an icon operating system under the ternary check of the voice screen mouse of the newly developed system is shown in Figure 1.
  • the time for which the icon is activated by voice should be determined according to the frequency of system operation and related operation management regulations.
  • the shadow system is provided with a shadow icon on the screen that is exactly the same size as the original system and has a shadow mouse parking area. Use the shadow icon to replace the corresponding original system icon, and establish the associated information of the voice and the shadow icon as in the newly developed system. At the same time, set the shadow mouse parking area when the mouse is not operating.
  • the shadow system is independently developed and runs independently. It has no impact on the operation of the original system and is only used for voice, icon and mouse operation verification. The workflow of the icon operating system at one time is consistent with that shown in FIG. 1.
  • Figure 3 is a schematic diagram of a double-line automatic blocking railway station console. The operator controls the switch and signal opening by operating the buttons on the screen. If the operation is incorrect, the train may enter the wrong track or drive in the wrong direction.
  • the forward entry button on the console is also the reverse exit button, and the reverse entry button is also the forward exit button. Therefore, the established voice and button association information table has one-to-one and one-to-many two. This situation is shown in Table 1.
  • buttons and voice may be complicated, which brings inconvenience to the operation.
  • the railway control console is operated independently by left and right, up and down, except for one pass operation. It is not necessary to distinguish between the upstream and downstream voice.
  • the corresponding relationship can be simplified into Table 2 by combining with relevant railway operating standards.
  • Table 2 Simplified buttons and voice-related information
  • the screen icon can be activated for each voice through the voice recognition system.
  • the "stop” voice can activate the X and S buttons; the "1 lane” voice can activate the X and I buttons.
  • the security operation settings of the mouse are set to the first level, the mouse is activated and an alarm is issued, and it cannot be operated.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • General Engineering & Computer Science (AREA)
  • Human Computer Interaction (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Audiology, Speech & Language Pathology (AREA)
  • Health & Medical Sciences (AREA)
  • Multimedia (AREA)
  • General Health & Medical Sciences (AREA)
  • Computational Linguistics (AREA)
  • Mechanical Engineering (AREA)
  • Acoustics & Sound (AREA)
  • User Interface Of Digital Computer (AREA)
  • Digital Computer Display Output (AREA)

Abstract

语音屏幕鼠标校核下的图标安全操作方法，通过操作者发出语音所关联的图标、屏幕上显示用来指定操作的图标和鼠标停止移动时所指向的图标进行校核比对，由系统给出操作的权限和警告，其基本特征是首先建立特定语音与图标的关联，通过语音识别系统的语音输入激活相关联图标，再与鼠标停止移动时所指向的图标进行校核比对。针对触摸屏计算机和移动终端而言，通过操作者发出语音所关联的图标、屏幕上显示用来指定操作的图标进行二元校核，由系统给出操作的权限。此外，还能针对已有系统提供影子系统进行防护。本发明利用现代语音识别工具，将人的行为通过电脑系统进行校核，最大限度确保操作正确，避免事故发生。

Description

语音屏幕鼠标校核下的图标安全操作方法 技术领域
本发明属于安全技术领域，具体涉及语音屏幕鼠标校核下的图标安全操作方法。
背景技术
随着计算机和互联网技术的发展，通过操作计算机和移动终端幕上的可激活图标来控制其所对应的后台设备运行已是生产和生活中的普遍现象。然而，由于受到种种限制，计算机和手机屏幕的大小总是有限的，相关应用软件开发人员只能在有限的空间内布置操控图标。这就必然导致用户在使用中出现大量的误操作。
误操作有两种：一种是所做的操作与操作人员当时的主观愿望是一致的，只是客观上导致了不良后果；二是所做的操作与操作人员当时的主观愿望是不一致的，只是因为在屏幕上误击所致。第一种误操作与设备无关，在技术设备上也无法防止的，完全由人的技术业务水平所决定。第二种误操作是人机交互过程中的错误，既有人的操作不认真的因素，也有设备操作不方便的因素。本发明创造针对的是第二种误操作。
在现有条件下，减少人机交互过程中的误操作有两种途径。一是加强操作人员的培训，提高操作人员的业务素质，严格规章制度，努力确保操作人员在操作前认真确认。这种途径属于管理问题。二是在系统设计时增加操作的冗余度，通过再度确认提示或双人甚至多人操作校核来确保操作的正确无误。这种途径属于技术途径。
现有技术途径存在以下问题：
1、再度“确认”虽然给了操作人员再次审核的机会，但由于操作人员过于自信，点击图标确认过快，错误确认仍时有发生。另外，对于频次较高的操作，再度确认不仅增加了操作的时间和难度，而且不断的确认也会使操作人员产生倦怠而失去警惕。特别是对时间要求比较高的高频操作，如铁路行车指挥，频繁确认是不可行的。
2、双人或多人操作校核的准确性较高，但其操作流程繁琐，需要人员较多，增加了操作人员间的联系时间，及时性无法保证，也会增加人力成本。
发明内容
本发明针对现有技术中的不足，提供语音屏幕鼠标校核下的图标安全操作方法，来减少人机交互过程中错误。所谓语音、屏幕和鼠标三元校核是指操作人员口说内容所关联的“语音图标”、屏幕上显示用来指定操作的“屏幕图标”和鼠标停止移动时所指向的“鼠标图标”必须完全一致，否则系统就会发出报警或禁止操作。
为实现上述目的，本发明采用以下技术方案：
一种语音屏幕鼠标三元校核下的图标安全操作方法，其特征在于，通过操作者发出语音所关联的图标、屏幕上显示用来指定操作的图标和鼠标停止移动时所指向的图标进行校核比对，由系统给出操作的权限和警告；首先建立特定语音与图标的关联，通过语音识别系统的语音输入激活相关联图标，再与鼠标停止移动时所指向的图标进行校核比对。
为优化上述技术方案，采取的具体措施还包括：
具体包括如下步骤：
步骤一、语音与屏幕图标的关联设置：设置语音与屏幕图标的对应关系，构建语音图标；
步骤二、关联语音识别系统：当系统接收特定的语音后，会激活所对应的屏幕图标；
步骤三、鼠标与屏幕图标的关联设置：当鼠标停留在未被语音激活的图标区域时，根据安全生产的等级要求进行鼠标与屏幕图标的关联设置；
步骤四、设置鼠标无操作时的屏幕区域：在屏幕上设置鼠标无操作时的停放区域。
所述步骤一中，语音和屏幕图标的关联一一对应、多对一或者一对多。
所述步骤二中，根据语音与屏幕图标的关联信息设置，某一特定的语音可以激活一个或多个屏幕图标。
所述步骤三中，鼠标和屏幕图标的关联设置根据安全生产的等级要求，设置成两级：第一级，鼠标激活并发出报警，禁止操作；第二级，鼠标激活并发出报警，但可以操作；当鼠标停留在被语音激活的图标区域时，可正常操作。
所述步骤四中，鼠标放入停放区域时，语音识别系统自动关闭语音接收功能；鼠标无操作时，未放入停放区域超过规定的设置时间，该区域会显示报警。
进一步地，还提出了一种语音屏幕鼠标三元校核下的图标安全操作方法，其特征在于，针对已有系统提供影子系统进行防护，影子系统是在原系统操作屏幕上设有与原系统大小完全相同且一一对应的影子图标，利用影子图标代替所对应的原系统图标，建立起语音和影子图标的关联信息；其中，影子系统独立运行，对原系统的运行无影响，只用于语音、图标和鼠标的操作校核。
进一步地，还提出了一种语音屏幕二元校核下的图标安全操作方法，其特征在于，针对触摸屏计算机或触摸屏移动终端，通过操作者发出语音所关联的图标、屏幕上显示用来指定操作的图标进行二元校核，由系统给出操作的权限。
进一步地，还提出了一种语音屏幕二元校核下的图标安全操作方法，其特征在于，针对触摸屏计算机或触摸屏移动终端，对已有系统提供影子系统进行防护，影子系统是在原系统 操作屏幕上设有与原系统大小完全相同且一一对应的影子图标，利用影子图标代替所对应的原系统图标，建立起语音和影子图标的关联信息；其中，影子系统独立运行，对原系统的运行无影响，只用于语音和图标的操作校核。
本发明的有益效果是：
1、针对目前计算机和移动终端屏幕操作存在的误操作问题，本发明提供了语音、屏幕和鼠标三元校核安全操作的方法，减少人机交互过程中错误。特别是在高安全领域，要求在操作前严格认真确认，做到“眼、手、口”一致，但这都严重依赖人的素质。本方法利用现代语音识别工具，将人的行为通过电脑系统进行校核，最大限度确保操作正确，避免事故发生。
2、为了对已有系统进行安全操作保护，本发明提出了“影子系统”概念和方法。由于影子系统是独立开发独立运行的，只用于语音、屏幕和鼠标的操作校核，对原系统的运行无任何影响。这给已有系统提供独立的安全操作保护软件提供了可能，且便于开发，更具实用性。
附图说明
图1是三元校核下一次图标操作的系统工作流程图。
图2是二元校核下一次图标操作的系统工作流程图。
图3是某双线自动闭塞铁路车站控制台示意图。
具体实施方式
现在结合附图对本发明作进一步详细的说明。
本发明创造实现的技术方案分为新开发系统和已有系统两种。
一、新开发系统
对于新开发系统在编程时需要增加以下工作：
1、设置语音与屏幕图标的关联信息
所谓设置语音与屏幕图标的关联信息就是设置语音与屏幕图标的对应关系，构建语音图标。语音与图标的关联原则上是一一对应的关系，但系统较为复杂时，强调一一对应关系，可能使语音变得过于复杂，给操作带来不便。因此，可根据实际应用情况、语音复杂程度和安全等级要求，设计成多对一和一对多。
2、关联语音识别系统
关联语音识别系统是指当系统接收特定的语音后会激活所对应的屏幕图标。根据语音与屏幕图标的关联信息设置，某一特定的语音可以激活一个屏幕图标，也可激活多个屏幕图标。
3、鼠标与屏幕图标的关联设置
当鼠标停留在未被语音激活的图标区域时，鼠标与图标的关联设置可根据安全生产的等级要求，设置成二级。第一级，鼠标激活并发出报警，不可以操作，这是最安全的设置；第二级报警可操作设置，即鼠标激活并报警，但可以操作，此时提醒操作人员加强确认。当鼠标停留在被语音激活的图标区域时，可正常操作。
4、设置鼠标无操作时的屏幕区域
在显示屏上相对空白的区域设置鼠标无操作时的停放区域，本设置是规范无操作时的鼠标停放。同时，鼠标放入停放区时，语音系统会自动关闭语音接收功能，尽可能减少外来声音对系统的干扰。鼠标无操作时未放入停放区域超过规定的设置时间，该区域会显示报警。
新开发系统的语音屏幕鼠标三元校核下的一次图标操作系统工作流程如图1所示，图标被语音激活所维持的时间应根据系统操作的频次和有关操作管理规定来确定。
二、已有系统
已有系统由于在开发时未进行语音、图标和鼠标的有关关联设置，所以校核时必须脱离开原系统，建立与原系统相对应的“影子系统”。
影子系统在屏幕上设有与原系统大小完全相同且一一对应影子图标，并增加影子鼠标停放区域。利用影子图标代替所对应的原系统图标，并如新开发系统一样，建立语音与影子图标的关联信息。同时设置好鼠标无操作时的影子鼠标停放区域。影子系统是独立开发，独立运行的，对原系统的运行无任何影响，只用于语音、图标和鼠标的操作校核。其一次图标操作系统工作流程与图1所示一致。
对于触摸屏计算机和移动终端而言，由于没有了鼠标，只有操作者发出语音所关联的图标、屏幕上显示用来指定操作的图标二元校核，无图标与鼠标的关联设置及鼠标停放区域。当二元校核不匹配时系统给出禁止操作。其一次图标操作系统工作流程如图2所示。
下面以铁路车站信号楼接发列车控制台操作来说明本发明的运用。
图3为某双线自动闭塞铁路车站控制台示意图。操作人员通过操作屏幕上的按钮，来控制道岔转换和信号开放，如果操作错误可能导致列车进入错误股道或开往错误的方向。
1、设置语音与操作按钮的关联信息
由于控制台上的正方向进站按钮同时也是反方向出站按钮，反方向进站按钮同时也是正方向出站按钮，故建立起来的语音与按钮关联信息表存在一对一和一对多两种情况，见表1。
表1 按钮与语音关联信息
序号 关联语音 按钮名称
1 下行进站、上行反方向出站 X
2 上行进站、下行反方向出站 S
3 下行反方向进站、上行出站 XF
4 上行反方向进站、下行出站 SF
5 下行1道 X I
6 上行1道 S I
7 下行2道 XII
8 上行2道 S II
9 下行3道 X3
10 上行3道 S3
11 下行4道 X4
12 上行4道 S4
在按钮较多的情况下，按钮与语音关联可能比较复杂，给操作带来不便。考虑到铁路控制台除了一次办理通过作业外都是以左右按上下行端分开独立操作的。语音区分上下行在实际操作中没有必要。结合铁路有关作业标准可将对应关系可简化成表2。
表2 按钮与语音关联信息简化
序号 关联语音 按钮名称
1 进站、反方向出站 X、S
2 反方向进站、出站 XF、SF
3 1道 X I、S I
4 2道 X II、S II
5 3道 X3、S3
6 4道 X4、S4
2、关联语音识别系统
根据语音与屏幕图标的关联信息，通过语音识别系统对每一种语音可以激活屏幕图标进行设置。比如“进站”语音可以激活X按钮和S按钮；“1道”语音可以激活X I按钮和S I按钮。
3、鼠标与按钮关联设置
当鼠标停留在未被语音激活的按钮区域时，将鼠标的安全操作设置均设置第一级，鼠标激活并发出报警，不可以操作。
4、设置鼠标无操作时的屏幕区域
在显示屏上相对空白的区域设置鼠标无操作时的停放区域，如图3中虚线圆圈区域。鼠标放入虚线圆圈区域时，语音系统会自动关闭语音接收功能。鼠标无操作时未放入停放区域超过规定的设置时间，该区域会显示报警。
以上是对新开发系统的举例。对于已有系统需建立与原系统相对应的“影子系统”，并独立运行。
需要注意的是，发明中所引用的如“上”、“下”、“左”、“右”、“前”、“后”等的用语，亦仅为便于叙述的明了，而非用以限定本发明可实施的范围，其相对关系的改变或调整，在无实质变更技术内容下，当亦视为本发明可实施的范畴。
以上仅是本发明的优选实施方式，本发明的保护范围并不仅局限于上述实施例，凡属于本发明思路下的技术方案均属于本发明的保护范围。应当指出，对于本技术领域的普通技术人员来说，在不脱离本发明原理前提下的若干改进和润饰，应视为本发明的保护范围。

Claims (9)

  1. 一种语音屏幕鼠标三元校核下的图标安全操作方法，其特征在于，通过操作者发出语音所关联的图标、屏幕上显示用来指定操作的图标和鼠标停止移动时所指向的图标进行校核比对，由系统给出操作的权限和警告；首先建立特定语音与图标的关联，通过语音识别系统的语音输入激活相关联图标，再与鼠标停止移动时所指向的图标进行校核比对。
  2. 如权利要求1所述的一种语音屏幕鼠标三元校核下的图标安全操作方法，其特征在于：具体包括如下步骤：
    步骤一、语音与屏幕图标的关联设置：设置语音与屏幕图标的对应关系，构建语音图标；
    步骤二、关联语音识别系统：当系统接收特定的语音后，会激活所对应的屏幕图标；
    步骤三、鼠标与屏幕图标的关联设置：当鼠标停留在未被语音激活的图标区域时，根据安全生产的等级要求进行鼠标与屏幕图标的关联设置；
    步骤四、设置鼠标无操作时的屏幕区域：在屏幕上设置鼠标无操作时的停放区域。
  3. 如权利要求2所述的一种语音屏幕鼠标三元校核下的图标安全操作方法，其特征在于：所述步骤一中，语音和屏幕图标的关联一一对应、多对一或者一对多。
  4. 如权利要求2所述的一种语音屏幕鼠标三元校核下的图标安全操作方法，其特征在于：所述步骤二中，根据语音与屏幕图标的关联信息设置，某一特定的语音可以激活一个或多个屏幕图标。
  5. 如权利要求2所述的一种语音屏幕鼠标三元校核下的图标安全操作方法，其特征在于：所述步骤三中，鼠标和屏幕图标的关联设置根据安全生产的等级要求，设置成两级：第一级，鼠标激活并发出报警，禁止操作；第二级，鼠标激活并发出报警，但可以操作；当鼠标停留在被语音激活的图标区域时，可正常操作。
  6. 如权利要求2所述的一种语音屏幕鼠标三元校核下的图标安全操作方法，其特征在于：所述步骤四中，鼠标放入停放区域时，语音识别系统自动关闭语音接收功能；鼠标无操作时，未放入停放区域超过规定的设置时间，该区域会显示报警。
  7. 一种语音屏幕鼠标三元校核下的图标安全操作方法，其特征在于，针对已有系统提供影子系统进行防护，影子系统是在原系统操作屏幕上设有与原系统大小完全相同且一一对应的影子图标，利用影子图标代替所对应的原系统图标，建立起语音和影子图标的关联信息；其中，影子系统独立运行，对原系统的运行无影响，只用于语音、图标和鼠标的操作校核。
  8. 一种语音屏幕二元校核下的图标安全操作方法，其特征在于，针对触摸屏计算机或触摸屏移动终端，通过操作者发出语音所关联的图标、屏幕上显示用来指定操作的图标进行二元校核，由系统给出操作的权限。
  9. 一种语音屏幕二元校核下的图标安全操作方法，其特征在于，针对触摸屏计算机或触摸屏移动终端，对已有系统提供影子系统进行防护，影子系统是在原系统操作屏幕上设有与原系统大小完全相同且一一对应的影子图标，利用影子图标代替所对应的原系统图标，建立起语音和影子图标的关联信息；其中，影子系统独立运行，对原系统的运行无影响，只用于语音和图标的操作校核。
PCT/CN2019/092714 2018-06-29 2019-06-25 语音屏幕鼠标校核下的图标安全操作方法 WO2020001421A1 (zh)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US17/256,650 US11656738B2 (en) 2018-06-29 2019-06-25 Secure operation method for icon based on voice-screen-mouse verification

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201810713169.X 2018-06-29
CN201810713169.XA CN109189280B (zh) 2018-06-29 2018-06-29 语音屏幕鼠标校核下的图标安全操作方法

Publications (1)

Publication Number Publication Date
WO2020001421A1 true WO2020001421A1 (zh) 2020-01-02

Family

ID=64948819

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2019/092714 WO2020001421A1 (zh) 2018-06-29 2019-06-25 语音屏幕鼠标校核下的图标安全操作方法

Country Status (3)

Country Link
US (1) US11656738B2 (zh)
CN (1) CN109189280B (zh)
WO (1) WO2020001421A1 (zh)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109189280B (zh) * 2018-06-29 2022-04-01 南京铁道职业技术学院 语音屏幕鼠标校核下的图标安全操作方法

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20140258855A1 (en) * 2013-03-06 2014-09-11 Nuance Communications, Inc. Task assistant including improved navigation
CN106201312A (zh) * 2016-06-30 2016-12-07 北京奇虎科技有限公司 一种应用处理方法、装置及终端
CN107657953A (zh) * 2017-09-27 2018-02-02 上海爱优威软件开发有限公司 语音控制方法及系统
CN107748602A (zh) * 2017-11-15 2018-03-02 安徽声讯信息技术有限公司 一种基于语音识别的智能语音计算机
CN109189280A (zh) * 2018-06-29 2019-01-11 南京铁道职业技术学院 语音屏幕鼠标校核下的图标安全操作方法

Family Cites Families (22)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5386494A (en) * 1991-12-06 1995-01-31 Apple Computer, Inc. Method and apparatus for controlling a speech recognition function using a cursor control device
JP2007127895A (ja) * 2005-11-04 2007-05-24 Nissan Motor Co Ltd 音声入力装置及び音声入力方法
JP4805861B2 (ja) * 2007-02-20 2011-11-02 株式会社バッファロー ワンセグデータ放送受信装置、ワンセグデータ放送受信方法、およびワンセグデータ放送受信プログラムを記録した媒体
JP2013070303A (ja) * 2011-09-26 2013-04-18 Kddi Corp 画面への押圧で撮影が可能な撮影装置、撮影方法及びプログラム
CN103186231A (zh) * 2011-12-30 2013-07-03 上海博泰悦臻电子设备制造有限公司 语音鼠标装置
US20150319170A1 (en) * 2012-12-21 2015-11-05 Didier Grossemy Computer implemented frameworks and methodologies for enabling identification verification in an online environment
US9858039B2 (en) * 2014-01-28 2018-01-02 Oracle International Corporation Voice recognition of commands extracted from user interface screen devices
TWI502481B (zh) * 2014-03-05 2015-10-01 Acer Inc 電子裝置以及使用者介面控制方法
KR20150145677A (ko) * 2014-06-20 2015-12-30 주식회사 퓨처플레이 사용자 인터페이스를 제공하기 위한 방법, 시스템 및 비일시성의 컴퓨터 판독 가능한 기록 매체
US9552816B2 (en) * 2014-12-19 2017-01-24 Amazon Technologies, Inc. Application focus in speech-based systems
US10156978B2 (en) * 2015-03-31 2018-12-18 Lg Electronics Inc. Terminal and operating method thereof
CN106168895A (zh) * 2016-07-07 2016-11-30 北京行云时空科技有限公司 用于智能终端的语音控制方法及智能终端
KR20180037850A (ko) * 2016-10-05 2018-04-13 삼성전자주식회사 긴급 서비스 제공 방법, 이를 위한 전자 장치 및 컴퓨터 판독 가능한 기록 매체
CN106898349A (zh) * 2017-01-11 2017-06-27 梅其珍 一种语音控制计算机的方法和智能语音助手系统
US20180321807A1 (en) * 2017-05-06 2018-11-08 Servicenow, Inc. Systems and methods for tailored content provision
DE102017211062B4 (de) * 2017-06-29 2022-06-30 Bayerische Motoren Werke Aktiengesellschaft Bedienelement für ein Fahrzeug
EP3438828B1 (en) * 2017-08-03 2019-12-11 Hitachi Rail Sts S.P.A. Method and system for remote control of human machine interfaces
JP6628775B2 (ja) * 2017-08-09 2020-01-15 双葉電子工業株式会社 タッチパネルスイッチ装置
CN109981878B (zh) * 2017-12-28 2021-09-14 华为终端有限公司 一种图标管理的方法及装置
CN108364645A (zh) 2018-02-08 2018-08-03 北京奇安信科技有限公司 一种基于语音指令实现页面交互的方法及装置
CN110007826A (zh) 2019-04-12 2019-07-12 深圳市语芯维电子有限公司 语音控制光标移动的方法和装置
CN210822020U (zh) 2019-09-10 2020-06-23 桂林师范高等专科学校 远近光灯自动切换装置

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20140258855A1 (en) * 2013-03-06 2014-09-11 Nuance Communications, Inc. Task assistant including improved navigation
CN106201312A (zh) * 2016-06-30 2016-12-07 北京奇虎科技有限公司 一种应用处理方法、装置及终端
CN107657953A (zh) * 2017-09-27 2018-02-02 上海爱优威软件开发有限公司 语音控制方法及系统
CN107748602A (zh) * 2017-11-15 2018-03-02 安徽声讯信息技术有限公司 一种基于语音识别的智能语音计算机
CN109189280A (zh) * 2018-06-29 2019-01-11 南京铁道职业技术学院 语音屏幕鼠标校核下的图标安全操作方法

Also Published As

Publication number Publication date
US11656738B2 (en) 2023-05-23
US20210263638A1 (en) 2021-08-26
CN109189280B (zh) 2022-04-01
CN109189280A (zh) 2019-01-11

Similar Documents

Publication Publication Date Title
US11621093B2 (en) Nuclear-power-plant computer-based procedure display device, safety control and monitoring system, and plant operation monitoring system
WO2020001421A1 (zh) 语音屏幕鼠标校核下的图标安全操作方法
CN108764691A (zh) 基于会签的审批节点在审批流程中的设置方法
TW201327361A (zh) 手機螢幕解鎖方法及系統
CN114065611A (zh) 适用于城市轨道交通的站内乘客疏散方法以及装置
CN108550029A (zh) 工作流审批节点按部门级别设置审批角色的方法
CN105292503B (zh) 一种安全高效的自动飞行控制系统控制显示机构
CN106682091A (zh) 一种无人机控制方法及装置
CN107808085A (zh) 智能终端的指纹控制方法及系统
CN106570684A (zh) 信息修改的方法及装置
CN112213971B (zh) Cbtc系统常态亮灯处理方法及装置
CN209911970U (zh) 一种厨房设备控制权限的管理系统
CN107301336A (zh) 基于表单时间性质字段的表单授权方法
CN107065819A (zh) 一种结合功能流程图的故障树建立方法
CN110412942A (zh) 一种用于白车身柔性工装切换自动防错系统
CN206317815U (zh) 一种复合屏蔽门系统
CN109445603A (zh) 一种基于数字沙盘的交互方法及数字沙盘
CN104657656B (zh) 一种基于银行系统的操控安全控制方法及装置
CN210199761U (zh) 一种无纸化考试授权装置
CN207129854U (zh) 一种起重机上下天车告知器
CN109358747A (zh) 陪伴机器人控制方法、系统、移动终端及存储介质
CN105280082B (zh) 电解机组操作培训系统
EP3640114A1 (en) Adjustable railway vehicle simulator and joint training simulator for railway driving-related workers
CN116450083A (zh) 一种对可激活图标实现语音图标校核操作的方法及装置
KR20200105030A (ko) 학생 승하차 확인 시스템 및 그 방법

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 19827233

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 19827233

Country of ref document: EP

Kind code of ref document: A1

122 Ep: pct application non-entry in european phase

Ref document number: 19827233

Country of ref document: EP

Kind code of ref document: A1

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 01.07.2021)

122 Ep: pct application non-entry in european phase

Ref document number: 19827233

Country of ref document: EP

Kind code of ref document: A1