JP2022088687A - Control device and control system - Google Patents

Abstract

To provide a control device which allows any operator to deal with an emergency by disabling an operation restriction when any abnormality occurs while restricting an unintended operation.SOLUTION: A control device which has an operation restriction of an operation object according to an operator and disables the operation restriction in a predetermined case comprises: an image processing function for detecting the operator and the operation object from an acquired image; a permission determination function for determining the presence/absence of the operation restriction when the operator operates the operation object; an abnormality detection function for detecting an abnormality; and an enabling/disabling control function for disabling the operation restriction regardless of the determination of the permission determination function when the abnormality detection function detects the abnormality.SELECTED DRAWING: Figure 2

Description

The present invention relates to a control device mounted on an automobile or the like, and more particularly to a control device that permits or prohibits an operation of an operation target depending on an operator and a situation.

Regarding the operation targets such as the steering wheel and the accelerator related to the driving of a car, a method has been proposed in which the persons who can operate the steering wheel and the accelerator are limited to specific target persons such as a driver, and unintended operations due to mischief by children are not accepted. For example, a method of generating an electric field in a passenger on the passenger side and locking the operation when the electric field is detected to prevent an unintended operation (see, for example, Patent Document 1), sitting in a specific seat. A method that allows only people to operate using gestures (see, for example, Patent Document 2), the operation is limited to those who have the key of the car, and if it is air conditioning, other people can easily control the air conditioning. A method for enabling this (see, for example, Patent Document 3) has been proposed.

Japanese Unexamined Patent Publication No. 2009-113694 Japanese Unexamined Patent Publication No. 2012-121386 Japanese Unexamined Patent Publication No. 2015-044441

However, in these methods, since the operation is restricted to a predetermined target person, this restriction cannot be lifted even in an emergency, and there is a problem that the danger cannot be avoided. That is, in the method of Patent Document 1, even if the driver cannot drive for some reason, the person in which the electric field is generated cannot operate the shift lever or the brake, so that the vehicle cannot be stopped or operated in an emergency. Further, since the person who can operate is specified by the presence or absence of the electric field, it is not possible to set the operation limit for each operation target or operator. In the method of Patent Document 2, the operator is recognized by image authentication, and the operation target for which the operation is restricted according to the preset permission level is automatically determined, but the permission level is set in advance or at the time of boarding. Since it is determined by authentication by gesture, it is impossible for an unauthorized passenger to immediately operate the brake or the like in an emergency. Further, in the method of Patent Document 3, since a key is required for authentication, it is not possible to set an operation restriction for each operation target person or operator. In addition, even in an emergency, a person who does not have the key cannot operate it to avoid the danger.

Therefore, the present invention has been made to solve the above-mentioned problems, and an object of the present invention is to provide a control device capable of allowing an arbitrary operator to respond to an emergency situation when some abnormality occurs. ..

The present invention is a control device that sets an operation limit for an operation target according to an operator and invalidates the operation limit in a predetermined case.
An image processing function that detects the operator and the operation target from the acquired image,
A permission judgment function that determines whether or not there are operation restrictions when the operator operates the operation target,
Anomaly detection function to detect anomalies and
It is a control device including an effective / invalid control function that invalidates an operation restriction regardless of the judgment of the permission determination function when the abnormality detection function detects an abnormality.

The present invention
With the above-mentioned control device,
A shooting device for acquiring images, and
A database that registers the operator and the operation restrictions set for the operator, and
A temperature sensor that measures the ambient temperature of the operator,
A weight sensor that measures the weight of the operator and
It is also a control system characterized by including an impact sensor that measures the impact received by the operator.

According to the present invention, operation restrictions are normally applied to each operator to prevent unintended operations of the operation target, and operation restrictions are invalidated in an emergency so that operations can be performed according to the situation.

This is an example in which the control device according to the first embodiment of the present invention is mounted on an automobile. It is a block diagram of the control device which concerns on Embodiment 1 of this invention. It is a flowchart which shows the operation of the control device which concerns on Embodiment 1 of this invention. It is a flowchart which shows the operation of the control device which concerns on Embodiment 1 of this invention. It is a block diagram of the control device which concerns on Embodiment 2 of this invention. It is a flowchart which shows the operation of the control device which concerns on Embodiment 2 of this invention. This is an example in which the control device according to the second embodiment of the present invention is mounted on an automobile. It is a block diagram of the control device which concerns on Embodiment 3 of this invention.

Embodiment 1.
FIG. 1 is an example in which the control device according to the first embodiment of the present invention is mounted on an automobile. FIG. 1A is a case where the front seat of the automobile is viewed from the front, and FIG. 1B is a front of the automobile. This is the case when the seat is viewed from the side.

In FIG. 1, two people, an operator (main) 10 who is a driver and an operator (secondary) 11 who is a passenger, are in a car. Inside the car, in addition to accelerators, brakes, parking brakes, shift levers, winkers, hazard switches, engine start keys or buttons, wiper and light control levers, air conditioning control, car navigation system, car audio, TV , Radio, door lock, power window open / close switch, seatbelt operation target 12 are installed.

Further, a photographing device (main) 13 such as a CCD camera is mounted in the vehicle. The photographing device (main) 13 can photograph the photographing range 14. The image taken by the photographing device (main) 13 is processed by the control device which is not visible in FIG. 1, and the operator and the operation target are detected. The control device also limits the operation of the operation target. The control device limits each operator's operation within a predetermined range, such as region (large) 15 or region (small) 16. In FIG. 1, the photographing apparatus (main) 13 is installed in the upper part of the front surface, but the position where the photographing apparatus (main) 13 is installed is not limited to this as long as the photographing range 14 can be photographed. For example, it may be installed in front of the operator 10, the lower part, the feet, or the like.

In FIG. 1, the operation limit is set for each area of a certain size such as the area (large) 15 and the area (small) 16, but it is also possible to set the operation limit for each operation target 12. It is also possible to provide a plurality of areas (small) 16 for each operation target 12.

FIG. 2 is a block diagram of the control device according to the first embodiment of the present invention, which is represented by 100 as a whole. 3 and 4 are flowcharts showing the operation of the control device 100, in particular, FIG. 3 shows the operation of the image processing function 102, and FIG. 4 shows the operation of the permission determination function 108. The operation of the control device 100 will be described with reference to FIGS. 2 to 4.

[Image processing function 102]
The image processing function 102 operates according to the flows 201 to 205 as shown in the flowchart of FIG. 3, and determines an abnormality of the operator. Each flow will be described below.

Flow 201: The image processing function 102 acquires an image taken by the photographing device (main) 13.

Flow 202: Based on the image data acquired in Flow 201, the person detection function 103, the location detection function 104, and the operation target detection function 105 are used to detect who, where, and what operation. The person detection function 103 detects whether or not a person exists, and if a person is further detected, checks whether or not the detected person has been pre-registered with the permission determination database 109. The location detection function 104 detects where the detected person is located. In the case of an automobile, a person seated in the driver's seat is detected as an operator (main) 10, and a person seated in another place is detected as an operator (secondary). The operation target detection function 105 detects what the detected person is operating. Information on "who", "where", and "what is being operated" detected by these three detection functions is output to the permission determination function 108.

Flow 203: The abnormality detection function 106 determines the presence or absence of an abnormality from the image acquired in the flow 201. If it is determined that there is an abnormality, the process proceeds to flow 204. On the other hand, if it is determined that there is no abnormality, the process proceeds to Flow 205. For example, if no person can be detected in the driver's seat, the operator (main) 10 does not look forward, or the operator (main) 10's eyes remain open, the operator (main) 10 has eyes. If it is left closed, it is judged that there is an abnormality.

Flow 204: When it is determined that there is an abnormality, the operation restriction is invalidated by using the operation restriction valid / invalid control function 107.

Flow 205: When it is determined that there is no abnormality, the operation restriction is maintained in the valid state by the operation restriction valid / invalid control function 107.

After the flow 204 or the flow 205 is completed, the process returns to the flow 201, and the flow 201 to the flow 204/205 are repeated again.

[Permission determination function 108]
The permission determination function 108 operates according to the flows 221 to 224 as shown in the flowchart of FIG. 4, and determines an abnormality in the operation of the operation target. Each flow will be described below.

Flow 221: Monitor the change of the input from the operation target 12. If it is determined that there is a change due to some operation by the operation target 12, the process proceeds to the flow 222, and if there is no change, the monitoring of the change is continued.

Flow 222: It is determined whether or not the operation restriction is invalidated. The case where the operation restriction is invalid means, for example, the case where some abnormality has already been detected and the operation restriction is invalidated. If the operation restriction is already invalidated, all the operators are allowed to operate the operation target as it is. If the operation restriction is valid, the process proceeds to flow 223.

Flow 223: The information of "who" input from the image processing function 102 "who is operating what" at "where" is collated with the information of the permission determination database 109, and the operation of the operation target 12 is permitted. , Or prohibit. If permitted, the operation target 12 can be operated. On the other hand, if prohibited, the process proceeds to flow 224, the state of the operation target 12 is maintained, and the operation remains impossible (locked). If an operator who is not registered in the permission determination database 109 is detected, it is determined as a guest.

If the operation is permitted in the flow 223, or after the flow 224 is completed, the flow returns to the flow 221 and the flow is repeated.

[Permission determination database 109]
In the permission determination database 109, as shown in Table 1, the registration ID of a person who can be an operator and the person who has each registration ID become an operator (main) or an operator (secondary). , Permission target, permission judgment method at the time of conflict, etc. are registered in advance. For example, cases where A, B, and C (A and B are parents and C are children) are family members and D is another person are applicable.

Table 1

In Table 1, the symbols b, c, and d are operation targets 12 such as brakes. When there are a plurality of operation targets 12, a plurality of operation targets 12 are specified as the area 1 including them. If permission is prioritized at the time of conflict, when multiple operators conflict, all of them are permitted to operate if even one of them is permitted to operate. On the contrary, when the time of conflict is the prohibition priority, if even one of the competing operators is prohibited from operating, all the operations are prohibited. If the operator with priority on permission and the operator with priority on prohibition conflict with each other, the setting at the time of conflict is treated as permission priority.

A control setting change function 110 mounted on a car navigation system, a smartphone, or a dedicated input device is used for registration in the permission determination database 109. An input device equipped with the control setting change function 110 may also be subject to operation restrictions.

As described above, in the control device 100 according to the first embodiment of the present invention, the operator (secondary) limits unintended operations and invalidates the operation restrictions when an abnormality is detected. Deputy) enables emergency response.

Embodiment 2.
FIG. 5 is a block diagram of the control device according to the second embodiment of the present invention, which is represented by 300 as a whole. The control device 100 of the first embodiment includes a temperature measuring device 301 for measuring the ambient temperature of the control device 300 (or the operator 10), a weight measuring device 302 for measuring the weight of the operator 10, and a control device 300. It has an impact measuring device 303 for measuring the impact of the device, an abnormality forcible setting device 304 for forcibly making the control device 300 into an abnormal state, and further, these devices 301 to 304 and an abnormality detecting function. It is different from having an abnormality determination function 305 for determining the presence or absence of an abnormality, a threshold value for abnormality determination, and a control setting change function 306 for setting on / off of each abnormality determination based on the input from 106. For the abnormality forced setting device 304, the abnormality determination is always on, and the abnormality determination is always performed.

That is, in addition to the configuration of the control device 100, the control device 300 includes a temperature measuring device 301, a weight measuring device 302, an impact measuring device 303, and an abnormality forced setting device 304. The temperature measuring device 301 measures the ambient temperature of the control device 300. For example, it detects a fire or the like. The weight measuring device 302 measures the weight of the operator. For example, it is used to estimate the age of the operator. The impact measuring device 303 measures the impact of the control device 300. For example, it detects a rear-end collision in a car. The abnormality forced setting device 304 is forcibly put into an abnormal state by manual operation.

The information (measured value) detected by these devices 301 to 304 is input to the abnormality determination function 305. In addition to these information, the information detected from the image information is input to the abnormality determination function 305 from the abnormality detection function 106.

[Abnormality determination function 305]
The abnormality determination function 305 determines the presence or absence of an abnormality based on the measured values of temperature, weight, and impact input from the devices 301 to 303 and the information such as the threshold value input by the control setting change function 306. .. Even when an abnormality is detected by the abnormality detection function 106 of the image processing function 102, it is determined that there is an abnormality. If the operator intentionally wants to invalidate the operation restriction, he / she can forcibly determine that there is an abnormality by manually operating the abnormality forced setting device 304.

The control setting change function 306 sets on / off whether or not to determine an abnormality for each of the information detected by the devices 102 and 301 to 303. For example, if the temperature abnormality determination is off, it is always determined that there is no abnormality regardless of the temperature measurement value and the threshold value setting.

FIG. 6 is a flowchart of abnormality determination in the abnormality determination function 305. Hereinafter, the operation of the abnormality determination function 305 will be described with reference to FIG.

Flow 401: The input from each of the above-mentioned devices 301, 302, 303, 304 and the abnormality detection function 106 is acquired.

Flows 410, 420, 430, 440: It is confirmed whether or not the abnormality determination is turned on / off, that is, whether or not the abnormality determination itself is performed. The on / off of the abnormality determination is set in advance by the control setting change function 306. When the abnormality judgment setting is off, the abnormality judgment is not performed regardless of the presence or absence of the actual abnormality. As a result, there is no abnormality, and the process proceeds to the abnormal flag-off flows 411, 421, 431, and 441. On the other hand, when the abnormality determination is on, the process proceeds to flows 412, 422, 432, and 442 for determining the presence or absence of an abnormality.

Flows 421, 422, 432, 442, 462: The threshold value or range for determining an abnormality, which is preset by the control setting change function 306, is input from each device 301, 302, 303, 304 and the abnormality detection function 106. Is compared to determine the presence or absence of an abnormality. If it is determined that there is no abnormality, the process proceeds to the abnormal flag-off flows 411, 421, 431, 441, 461, and if it is determined that there is an abnormality, the process proceeds to the abnormal flag-on flows 413, 423, 433, 443, 463. ..

Flow 413, 423, 433, 443, 463: When an abnormality is found in the temperature, weight, impact, or image in the flow 421, 422, 432, 442, 462, or when a forced abnormality is manually input. Turn on the corresponding flag.

Flow 450: When any one of the forced abnormality flag-on 463 of the temperature abnormality flag-on 413, the weight abnormality flag-on 423, the impact abnormality flag-on 433, and the image abnormality flag-on 443 is recognized, the abnormality operation restriction is enabled according to the flow 451. When all the abnormality flags are off 411, 421, 431, 441, 461, the operation restriction is invalidated according to the flow 452. When the abnormal flow ends, the entire flow starts again. Other flows are the same as in the case of the first embodiment, but for example, 203, 204, and 205 in FIG. 3 are replaced with 442, 451 and 452 in FIG.

In the flowchart of FIG. 6, each abnormality determination is processed in parallel, but the present invention is not limited to this, and if all the determinations are completed before the flow 450 for determining the presence / absence of the abnormality flag on. good. For example, each abnormality determination may be processed in series.

As described above, in the control device 300 according to the second embodiment of the present invention, the operator (secondary) limits unintended operations and invalidates the operation restrictions when an abnormality is detected. Deputy) enables emergency response. In particular, by detecting an abnormality not only by the state of the operation target 12 but also by temperature, weight, impact, etc. and invalidating the operation restriction, a wider emergency response becomes possible.

Embodiment 3.
FIG. 7 shows an example in which the control device according to the third embodiment of the present invention is mounted on an automobile. FIG. 7A is a case where the front seat of the automobile is viewed from the front, and FIG. 7B is a front of the automobile. This is a case where the sheet is viewed from the side surface, and the same reference numerals as those in FIG. 1 indicate the same or equivalent points. Further, FIG. 8 is a block diagram of the control device according to the third embodiment of the present invention, which is represented by 500 as a whole.

As shown in FIG. 7, in the third embodiment of the present invention, a photographing device (secondary) 17 is added, and a photographing range 18 thereof is also added accordingly. In the control device 500 according to the third embodiment of the present invention, when the image pickup device (main) 13 alone cannot photograph the operator (main) 10, the operator (secondary) 11, and the operation target 12, the photographing device (secondary) The shooting range is expanded by using the above, and the operation target 12 is shot. For example, the photographing device (main) 13 detects the operator (main) 10 and the operator (secondary) 11 and the abnormal state is detected, and the photographing device (secondary) 17 operates the operation target 12 by which operator. The accuracy can be improved by specifying whether or not it is present.

As shown in FIG. 8, the image taken by the photographing device (secondary) 17 is input to the image processing function 102, and together with the image input of the photographing device (main) 13, the abnormality detection function 106, the person detection function 103, It is processed by the location detection function 104 and the operation target detection function 105.

As described above, in the control device 500 according to the third embodiment of the present invention, it is possible to acquire an image in a wider range, and it is possible to improve the detection accuracy of an abnormal situation.

In FIG. 7, the photographing device (main) 13 and the photographing device (secondary) 17 are arranged at the upper part, but the imaging device is not limited to these positions as in the first embodiment, and any of the photographing devices is used. May be placed behind the operator to photograph the operation target 12. Further, the number of photographing devices (secondary) 17 to be installed is not particularly limited. That is, if necessary for improving the detection accuracy of the operator and the operation target, a plurality of photographing devices (secondary) 17 may be installed.

Further, in FIG. 8, the control device 500 according to the third embodiment of the present invention has a configuration in which a photographing device (secondary) 17 is added to the control device 300 of the second embodiment, but the first embodiment has a configuration. It is not possible to add a photographing device (secondary) 17 to the control device 100.

In the first to third embodiments, the case where the control devices 100, 300, and 500 are mounted on the automobile has been described as an example, but the present invention is not limited to the automobile, and there are a plurality of possible operators. However, if you want to restrict the operation of each operator, it can be applied to other than automobiles. For example, it can be applied to the operation of airplanes, ships, trains, the operation of information terminals such as PCs, tablets, and smartphones, and the operation of processing machines such as NCs, lathes, and presses.

10 operator (main), 11 operator (secondary), 12 operation target, 13 shooting device (main), 14 shooting range, 15 operation permission area (large), 16 operation permission area (small), 100 control device , 102 image processing function, 103 person detection function, 104 location detection function, 105 operation target detection function, 106 abnormality detection function, 107 operation restriction valid / invalid control function, 108 permission judgment function, 109 permission judgment database, 110 control settings Change function.

Claims (9)

A control device that sets operation restrictions for operation targets according to the operator and invalidates the operation restrictions in a predetermined case.
An image processing function that detects the operator and the operation target from the acquired image, and
When the operator operates the operation target, a permission determination function for determining whether or not there is an operation restriction, and a permission determination function.
Anomaly detection function to detect anomalies and
A control device including an effective / invalid control function that invalidates the operation restriction regardless of the determination of the permission determination function when the abnormality detection function detects an abnormality. The permission determination function compares the operator and the operation target detected by the image processing function with the information registered for the operator and the operation target, and determines the presence or absence of the operation restriction. The control device according to claim 1. The control device according to claim 1, wherein the image processing function further detects the position of the operator. The control device according to claim 1, wherein the abnormality detection function detects an abnormality of the operator from an image of the operator detected by the image processing function. The first aspect of the present invention is characterized in that the abnormality detection function detects an abnormality of the operator based on at least one of the ambient temperature of the operator, the weight of the operator, and the impact received by the operator. The control device described. The control device according to claim 1, wherein the abnormality detection function can be forcibly set to an abnormal state. The control device according to claim 1, further comprising a control setting changing function for switching on / off of the abnormality detection function. The control device according to any one of claims 1 to 7, wherein the control device is mounted on an automobile. The control device according to any one of claims 1 to 8.
An imaging device for acquiring the image and
A database for registering the operator and the operation restrictions set for the operator, and
A temperature sensor that measures the ambient temperature of the operator,
A weight sensor that measures the weight of the operator,
A control system comprising an impact sensor for measuring the impact received by the operator.

Images