RU2111134C1 - Method of and system for providing vigilance of vehicle driver - Google Patents

Abstract

FIELD: mechanical and transport engineering; keeping vehicle drivers and operators in different branches of industry in fit-for-work state. SUBSTANCE: in proposed method adequate reaction of driver on sense of stimulus under reduction of vigilance level is analyzed. For this purpose sound stimulus in form of at least one speech question (command) to driver is formed to make the driver effect actively realized operation. Speech answer of driver to question (command) and (or) carrying out of above indicated operation is recorded as reaction. Then content of question (command) and speech report or carried out operation are compared and their conformity is taken as criterion of watchfulness. System has processor-controlled stimulus and command forming units, and unit to analyze reaction of driver to speech stimuli, including recognition of speech, and stimulus and reaction analyzing unit. EFFECT: provision of operative control of functional condition of operator. 5 cl, 1 dwg

Description

 The invention relates to means for maintaining the health of drivers of vehicles, as well as operators of various professions, and can be used on rail, road, water, air transport, in various industries for operational monitoring of the compliance of the functional state with the requirements of professional activity.

 The determination of a person’s wakefulness level, his attention, which characterizes professional activity, acquires special significance based on the increasingly growing requirements for a person as an operator making responsible decisions. The most numerous drivers of vehicles (TS), which requires the development of methods of hardware control with the allocation of an unacceptable physical and / or mental state. These methods are based on the monitoring of wakefulness by heart rate, brain and muscle activity, as well as other physiological functions, and the introduction of "feedback". Feedback makes these devices operational. Among the early inventions of this kind is a driver performance system with power sensors mounted in the steering wheel. When the muscles relax, which is characteristic of the process of falling asleep, the driver’s hands weaken. This triggers an automatic wake-up system: a stream of aromatic substance is directed into the driver's face (US 2199060, YOUNG, 340-279, 1940).

 According to the type of interaction with the operator, “passive” and “active” monitoring methods are distinguished.

 Among the "passive" methods that allow obtaining objective information about the psychoemotional state of a person in real time, is the registration of the current value of the physiological parameter. So, in the invention (SU 0938932 A, BELOV et al., A 61 B 5/05, 1982), the driver is equipped with a wrist bracelet with electrodes to record the value of the relative change in the electrical conductivity of the skin, as well as with a head-mounted electric stimulator controlled by a signal from the aforementioned a bracelet. For the purposes of rail transport, the driver's condition analyzer automatically controls the brakes and the alarm (SU 938932 A, LENMETROPOLITEN, B 61 L 3/08, B 60 T 7/14, 1987). A detailed analysis of the signals of the phase component of the skin-galvanic reaction (GGR) (RU 2025731 A1, BONCH-Bruevich et al., G 09 B 9/00, 1994) allows to increase the reliability of determining the level of wakefulness. The telemechanical operator’s wakefulness control system (TKSBM) (manufactured by AOZT NEYROKOM, Russian Federation, M, 1996), which uses phase separation of the RAG signals, implements biological feedback for controlling the driver’s wakefulness, which is mass-produced for rail transport.

 The “active” methods for controlling the vigilance of the driver (operator), providing for the stimulus and determining the response of the operator, received an independent direction. Based on the reaction time, the alarm is turned on or the vehicle is forced to stop. On locomotives, such a solution uses the “vigilance handle” (LS Nersesyan, Psychological Aspects of Improving Reliability of Control of Moving Objects, ed. Promedek, M., 1992, p. 125). Based on this principle, a driver alertness monitor was built (WO 96/16830 A1, WILLIAMS et. Al, B 60 K 28/06, 1996), while the reaction time is compared with the standard one taking into account the daily activity cycle and particular characteristics of the driver and, in this sense, the system interactive. The stimulus is given in the form of an audio signal. The response of the driver is recorded using a sensor; the entire system is controlled by the processor. The invention WO 96/16830 A1 may be adopted as a prototype for a patentable group of inventions.

 The objective of the invention is to develop principles and create a system for maintaining the health of the driver of the vehicle (operator) in a monotonic environment, combining the advantages of both types of the systems described above.

 The technical result of the invention consists in increasing the reliability of determining the current psychophysiological state of the driver (operator) and, accordingly, making an adequate situation for the decision to drive a vehicle. An additional technical result is seen in the fact that a forced dialogue between the driver and the system has an activating effect that impedes the development of psychophysiological relaxation and, especially, falling asleep.

 The technical result is ensured due to the fact that in the method of maintaining the driver’s health, the driver analyzes the adequacy of the driver’s response to the meaning of the stimulus in the face of a decrease in wakefulness, which creates a sound stimulus in the form of at least one speech question (command) to the driver for an actively conscious action . As a response, the driver’s voice message is recorded in response to a question (command) and (or) the fact that the above action is performed. Next, the contents of the question-command and the voice message or the performed action are compared, and their correspondence is taken as a performance criterion.

 The method can be characterized by the fact that a decrease in the level of wakefulness is recorded by a change in the physiological parameters of the pulse or (and) the skin-galvanic reaction of the driver, as well as by the fact that questions (commands) are formed in a random sequence.

 As an actively conscious action formulated in the question (team), it is possible to recommend the driver to act on auxiliary means of driving the vehicle, to report the characteristics of its functioning and control, the situation on the traffic area.

 The system for maintaining the operability of a vehicle driver includes means connected to the processor for supplying sound stimuli with a sound transducer, means for receiving and analyzing a driver’s response to sound stimuli with sensors, alarm means and a display. In addition, a unit for analyzing correspondence (stimulus and reaction with) connected to it with blocks for inputting current information and analyzing a driver’s response to stimuli and commands, a memory block for emergency situations, a block for generating stimuli and commands, has been introduced. There is a block for presenting stimuli and commands connected to a block of visual stimuli and a block for developing a decision on driver inoperability associated with the vehicle control unit. Means for supplying sound stimuli are made in the form of a block of speech stimulus synthesis, the output of which is connected to a sound transducer.

 Means for receiving and analyzing a driver’s response to sound stimuli comprise a microphone connected to a block for receiving and recognizing speech information. The sensors for working out the commands are connected to the input of the block for receiving the execution of commands, both of which are connected to the block for analyzing the driver’s response to stimuli and commands.

 The invention is illustrated in the drawing, which shows a block diagram of a system for maintaining the health of a driver of a vehicle that implements the patented method.

 The basis of the invention are the following premises. It is known that the processes of human wakefulness are strictly individual and subject to external and internal rhythms (Wayne A.M. Wakefulness and sleep, M., Science, 1970, p. 8-11). Monotonous activity, namely such a state prevails among the operators of the processes under consideration, causes a decrease in the level of wakefulness, and hence optimal performance (in particular, a decrease in the adequate and highly stable reaction of the system to external influence) (V.G. Volkov, V.M. Mashkova. Methods and devices for assessing the functional state and level of human performance, M., Science, 1993, p. 8). It is proposed to enter into the wakefulness control process speech dialogue operations in the "question-answer" mode, followed by an analysis of the adequacy of the answer to the given question. If the response is inadequate, a conclusion is drawn about a decrease in the level of wakefulness, which is proposed to be refuted by a conditional action (action on the handle of vigilance) or a speech response.

 It is noted that a change in the psychophysiological state is reflected in the speech signal of the subject, and this fact makes it possible to objectify and improve the accuracy of diagnosis. For example, the emotional state or inhibition of a person in dynamics is diagnosed by measuring the roughness of the fundamental tone in time (SU 793575 A, GALUNOV et al., A 61 B 5/16, 1981). Fatigue and stress for an individual operator can be determined by the energy parameters of the speech signal (SU 1459655 A1, LIAP, A 61 B 5/16, 1989), other characteristics of the arrhythmia of speech signals (SU 1581279 A1, VNIIMP, A 61 B 5/16, 1990).

 It is also known that during the transition from wakefulness to a drowsy state, in particular during overwork due to monotonously acting factors, thinking disorders are observed, which are expressed in the fact that human speech becomes inadequate to the situation. It is less legible, thoughts get confused, become incoherent, speech information is distorted or speech is completely absent. A connection is also known between speech arousal and changes in skin resistance (EP 0660303 A1, AT&T Corp., G 10 L 9/16, 1995). The above physiological features are the main criterion in the patented invention for making decisions on reducing the level of wakefulness. The very means for the synthesis and recognition of speech are known in science and technology, in particular, from the prior art, and are not the subject of the present invention.

 The implementation of the patented method is considered on the example of the functioning of the system for maintaining the health of the driver of the vehicle, a block diagram of which is shown in the drawing.

 The system comprises a current information input unit 1, a processor 2, a microphone 3 connected to a speech reception and recognition unit 4. The system includes sensors 5 working out commands for controlling the vehicle, connected to the unit 6 for receiving the execution of commands, an audio transducer 7 (loudspeaker, headphones) connected to the unit 8 for the synthesis of speech stimuli. There is a block 9 of visual stimuli connected to the block 10 of the presentation of stimuli and commands. Blocks 8 and 10 are connected with each other and with the block 11 of the formation of stimuli and commands connected to the processor 2. In addition, blocks 8 and 10, as well as 4 and 6 are connected to the block 12 of the response of the driver to the stimulus associated with block 13 analysis of the correspondence of the stimulus and reactions also controlled by processor 2.

 The system includes blocks 14 of the memory of emergency situations and block 15 of the decision to manage the vehicle, connected to the bodies 16 of the vehicle control. Block 16 is connected with block 17 enable signaling devices and block 18 emergency braking. For convenience, operational information from the processor 2 can be displayed on the display 19. The processor 2 on the corresponding buses provides synchronization of all units, the collection and processing of incoming information and the management of the entire system.

 The current information input unit 1 has a group of inputs connected to the corresponding vehicle parameter recorders. For devices with electric traction, this can be: a speedometer, manometers, voltmeters. For road vehicles, these are: speedometer, tachometer, various gauges - engine temperature, gas tank filling, oil pressure, air pressure in brakes, tires. In addition, there is a group of inputs that records the physiological parameters of the driver: a skin-galvanic reaction, as well as, for example, pulse, oxygenation, etc., depending on the specific type of vehicle.

 A group of inputs of external parameters of the road situation is provided, for locomotive drivers, for example, a tutor of traffic signals. For drivers of vehicles, for example, the distance to an obstacle or traffic obstruction, location (location coordinates), information from traffic signs, a television image of the rear and all-round visibility (the means are described - J. "Automotive Industry", 1996, N 9, p. 29-30), headlights of an oncoming vehicle, weather conditions.

In block 14 of the memory of emergency situations, the current categorization (duration) of driver states is introduced:
a. rapidly developing (death, loss of consciousness);
b. average speed of development (going to sleep, turning off attention, the so-called narrowing of the attention function);
in. slow (a state of deep fatigue, an increase in prodromal symptoms of painful conditions).

 Each category mentioned has its own objective parameters, respectively, by comparing with current values it is possible to forecast the immediate consequences and measures to prevent them.

 When conducting intelligent processing of information from blocks 1 and 14, processor 2, in accordance with the program, generates stimuli and commands in block 11, which are sent to block 8 of the synthesis of speech stimuli and block 10 of presentation of visual stimuli and commands.

 In the event that situation a occurs, the action of the system is the simplest: through the processor 2 and the decision-making block 15, a direct command is issued to block 16 to turn on emergency braking for railway vehicles or to take any action to immediately stop traffic for other vehicles (block 18) and to turn on the alarm (17).

 Patented method and system are designed primarily for work in situations b and especially c, as the most often leading to accidents. Consider in detail the implementation of the method in these conditions.

 First, the driver’s current state, vehicle parameters, external parameters are determined, they are compared with the parameters in block 14, and a forecast is made for the nearest time period (for example, for the next minute).

 The psychophysiological state can be assessed by the parameters of the phase component of the RAG using, for example, the above-mentioned TCSM device or the like. A criterion for a decrease in working capacity is an increase in the duration of the interval between pulses of the phase component of the RGR in relation to the norm.

 The parameters of the vehicle can be attributed indicators of speed, pressure, voltage, electric power (for devices with electric traction). For road vehicles, this is also information from a speedometer, tachometer, engine temperature, gas tank filling, oil pressure, air pressures in brakes, tires. External parameters are indicators of the traffic situation, traffic signals, weather conditions, etc., i.e. those indicators that can be determined by hardware.

 In the case of an unfavorable forecast, the processor 2 initiates a block 11 of the formation of stimuli and commands. This block 11, depending on the specific situation (the "severity" of the forecast) through the block 8 of the synthesis of speech stimuli and sound transducer 7, enters into voice communication with the driver. A team or one or more pre-formulated questions related to the driving process or the driver’s condition are asked. Moreover, the selection of these questions in each "question-answer" cycle is carried out randomly to eliminate addiction.

 The essence of the method does not change if the question is not formulated by speech, but through block 10 of the presentation of incentives and commands. This can be, like light information on the scoreboard, or information in another form, for example, mnemonic, forming a request for the technical condition of the means ("typhon", "pantograph", etc.).

 The response of the driver through the action goes directly to block 6, and in speech form is converted by the microphone 3 into an electrical signal, which is processed in the block 4 for receiving and recognizing speech information. In the event that the speech stimulus requires taking measures to control the vehicle and the driver performs such operations to execute commands by means of sensors 5, the signals are processed in block 6 for receiving the execution of vehicle control commands. Further, the information is processed by block 12, which determines the response of the driver to the stimulus.

 By comparing the information about the driver’s reaction, about the current situation from block 1, as well as from block 14, block 13 draws a conclusion about the correspondence of the concept of “wakefulness” if the semantic content of the question and answer correspond, which, all other things being equal, indicates operability the driver. In this case, the dialogue is terminated.

 If the semantic analysis in block 13 gives a negative result, taking into account the current situation and the information from the pre-emergency memory block 14, the processor 2 makes a conclusion about the impossibility of further driver work or gives a command to turn on the emergency system.

 There are many different options for implementing the system, depending on skill and professional knowledge, as well as the used elemental base and technical means. Therefore, the above block diagram should not be a limitation upon the implementation of the invention.

 According to the information available to the applicant, the group of inventions is new and according to the description is industrially applicable. It seems that the invention satisfies the condition of inventive step, as it is based on the knowledge established by the inventors. This knowledge boils down, firstly, to the statement of the problem of maintaining the driver’s health of the vehicle according to the criterion of semantic correspondence “question-answer”, and secondly, to the simultaneous activation of driver’s wakefulness through speech stimulation in a dialogue mode, which is very useful.

Claims (5)

 1. A method of maintaining the driver’s health of a vehicle, including generating a sound stimulus, registering a driver’s response to a stimulus, comparing the parameters of the said reaction with established criteria and generating an activation signal if the performance criteria do not meet, characterized in that they analyze the adequacy of the driver’s response to the meaning of the stimulus in conditions reduce the level of wakefulness, for which form a sound stimulus in the form of at least one speech question (command (s) the driver responds to an actively-conscious action, in response to the driver’s voice message in response to a question (command) and (or) the fact that the said action was performed, the contents of the question (command) and the voice message or performed action are compared and their correspondence is taken as a criterion health.  2. The method according to claim 1, characterized in that the decrease in the level of wakefulness is recorded by a change in the physiological parameters of the pulse or (and) the skin-galvanic reaction of the driver.  3. The method according to claim 1 or 2, characterized in that the questions (teams) are formed in random order.  4. The method according to any one of claims 1 to 3, characterized in that as an actively-conscious action formulated in the question (team), it is recommended that the driver act on the auxiliary means of driving the vehicle and (or) report the situation on the traffic area.  5. A system for maintaining the operability of a vehicle driver, including means for supplying sound stimuli with a sound transducer connected to the processor, means for receiving and analyzing the driver's response to sound stimuli with sensors, signaling means and a display, characterized in that additionally connected to the processor are introduced a unit for analyzing the correspondence of a stimulus and a reaction with blocks for inputting current information and an analysis of a driver’s response to stimuli and commands connected to it, a pre-emergency memory unit situations, the stimulus and command generation unit, as well as the stimulus and command presentation unit connected to the visual stimulus unit, and the driver inoperability decision unit associated with the vehicle control unit, while the means for supplying sound stimuli are made in the form of a synthesis unit speech stimuli, the output of which is connected to the sound transducer, means for receiving and analyzing the driver’s response to sound stimuli contain a microphone connected to the block for receiving and recognizing speech information rations, command processing sensors are connected to the input of the command execution reception unit, both of which are connected to the driver response analysis unit to stimuli and commands.