JP2018056890A - Automatic learning device, method, program, automatic learning system, and automatic monitor device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an automatic learning device, method, program, automatic learning system, and automatic monitor device that automatically learn a state of an apparatus to be controlled, by controlling the apparatus before observing a variation caused by the control.SOLUTION: An automatic learning device 10 for automatically learning a state of a remotely controllable apparatus comprises: a command transmission unit 18 for transmitting a control command for controlling a target apparatus into a specific state; a monitor unit 20 for monitoring at least a state of the target apparatus after transmitting the control command before acquiring monitor data as a result of the monitoring; and a learning unit 26 for performing learning on the state of the target apparatus on the basis of an apparatus state label showing the specific state of the target apparatus and the acquired monitor data associated with the apparatus state label, to create an apparatus state recognition model for giving a state of the target apparatus as output when receiving input of monitor data.SELECTED DRAWING: Figure 2

Description

  The present invention relates to an automatic learning apparatus, method, program, automatic learning system, and automatic monitoring apparatus that automatically learn the state of a remotely controllable device.

  In recent years, attention has been focused on Internet of Things (IoT) in the home. For example, remote control for a device that remotely turns on an air conditioner in a home has been put into practical use. However, since IoT requires a device compatible with a specific protocol such as ECHONET Lite and AllJoyn, it is not widely used in general households and is currently limited in use. On the other hand, an infrared remote controller is a common remote control device in a house that has already been widely used. Infrared remote controls are used in a wide range of devices such as televisions, air conditioners, fans, ceiling lights, etc., and control is performed using infrared remote controllers that can be remotely operated not only from within the house but also from outside the house on the flow of IoT. Yes. However, unlike the IoT, the infrared remote control is controlled by one-way traffic, so it cannot be determined whether the control is actually applied from outside the house or the state of the device.

  In order to acquire the state of the device to be controlled, for example, power is measured with a wattmeter or breaker attached to each device so that the operation can be confirmed with the power consumption. On the other hand, these devices are supposed to be operated from home, and are often designed so that the state of the device can be visually and audibly recognized by humans. Therefore, it is also effective to check the state of the device visually and auditorily from outside the house using the camera image.

  In particular, in image recognition, deep learning utilizing high-performance computing capability has attracted attention in recent years, and advanced recognition of the state of a device (Non-Patent Document 1) is possible. The image recognition based on this deep learning is different from the recognition technique based on the feature points based on the conventional manual design, and features are automatically extracted. However, a large amount of sample images are required for the learning.

  In order to recognize a plurality of devices from one image, Fast R-CNN (Region with Convolutional Neural Networks) (Non-Patent Document 2), which is a kind of deep learning, is generally used. However, this also requires a large amount of learning sample images in which the position of the device and the device state are labeled as to what part of the image is reflected.

Krizhevsky, Alex, Ilya Sutskever, and Geoffrey E. Hinton. "Imagenet classification with deep convolutional neural networks." Advances in neural information processing systems. 2012. Girshick, Ross. "Fast r-cnn." Proceedings of the IEEE International Conference on Computer Vision. 2015.

  In order to determine the state of a controlled device from an image, it is necessary to prepare and learn a sample image for learning in advance. Further, identification of the target, which is the device to be controlled, is also an issue. For example, when it is designated as an electric light, it is necessary to be able to recognize which electric light it is, and it is necessary to match the equipment designated on the control side with the equipment actually recognized from the image.

  For this reason, according to the conventional methods, it has been necessary to manually label and learn images taken in advance. Even if the state of a general device can be determined, if there are multiple devices of the same type in the image, it cannot be determined which is the control target and cannot be generalized. It was necessary to manually map the devices recognized from the images in advance.

  The present invention has been made in view of such circumstances, and an automatic learning device, method, which automatically learns the state of a controlled device by controlling the device and observing changes caused thereby, An object is to provide a program, an automatic learning system, and an automatic monitoring device.

  (1) In order to achieve the above object, the present invention has taken the following measures. That is, the automatic learning device of the present invention is an automatic learning device that automatically learns the state of a remotely controllable device, and includes a command transmission unit that transmits a control command for controlling the target device to a specific state, and at least Monitors the status of the target device after transmitting the control command, and obtains monitor data as a result of the monitoring; corresponds to a device status label indicating a specific status of the target device and the device status label A learning unit that performs learning on the state of the target device based on the acquired monitor data attached, and generates a device state recognition model that outputs the state of the target device as an output when the monitor data is input. It is characterized by providing.

  In this way, by using the data automatically collected by controlling and monitoring the target device in a specific state, a device state recognition model for recognizing the state of the device indicated by the monitor data can be automatically generated. Therefore, the labor of manually preparing sample data for learning can be saved. Also, accurate device control is possible based on the generated device state recognition model.

  (2) Moreover, the automatic learning device of the present invention further includes a model evaluation unit that evaluates the device state recognition model, and the model evaluation unit uses a part of the acquired monitor data as test data. Whether or not the state recognition model can correctly recognize the test data is evaluated, and generation of the device state recognition model is repeatedly performed until a correct answer rate obtained as a result of the evaluation reaches a predetermined value.

  Accordingly, an accurate device state recognition model can be generated, and more accurate device control can be performed based on the generated device state recognition model.

  (3) In the automatic learning device of the present invention, the learning unit selects monitor data having high similarity from the monitor data in the specific state as representative monitor data, and uses the representative monitor data as a learning target. The apparatus state recognition model is generated.

  Thereby, it is possible to remove monitor data indicating different states that are exceptionally mixed when control fails. Therefore, the learning target data does not include data that causes noise, which is one of the causes of a decrease in learning accuracy, so that highly accurate learning and a device state recognition model can be generated.

  (4) In the automatic learning device of the present invention, the monitor unit includes a camera, and acquires image data as the acquired monitor data. As a result, it is possible to generate a device state recognition model of a device in which a change in appearance occurs due to transmission of a control command.

  (5) Moreover, the automatic learning device of the present invention includes a position extraction unit that extracts a position of a device from a difference between monitor data with different device state labels associated with each other in the acquired monitor data. And Thereby, the state of the device including the position of the device can be learned. Further, even when there are a plurality of similar devices, these devices can be distinguished by their positions.

  (6) Moreover, the automatic learning apparatus of this invention WHEREIN: The said monitor part is equipped with a microphone, and acquires audio | voice data as said acquired monitor data. As a result, it is possible to generate a device state recognition model of a device in which a sound change occurs due to transmission of a control command, and it is possible to accurately control such a device.

  (7) The automatic monitoring device of the present invention is an automatic monitoring device that automatically monitors the state of a remotely controllable device, and transmits a control command for controlling the target device to a specific state. And a monitor unit that monitors at least the state of the target device after transmitting the control command and acquires monitor data as a result of the monitoring, and a device state label indicating a specific state of the target device; The acquired monitor data associated with the device status label is used for learning of the status of the target device in a separate automatic learning device, and the status of the target device when monitor data is input by the learning. Is generated as a device state recognition model. Thereby, the learning data of the target device can be automatically acquired, and the device state recognition model can be automatically generated using the device state label of the target device and the acquired monitor data.

  (8) Moreover, the automatic learning system of this invention is an automatic learning system which learns the state of the apparatus which can be controlled remotely, Comprising: The automatic monitor apparatus as described in (7), and the said object apparatus acquired A data acquisition unit that acquires a device state label indicating a specific state of the device and monitor data associated with the device state label, and learning of the state of the target device based on the acquired device state label and monitor data And a learning device having a learning unit that generates a device state recognition model that outputs the state of the target device as an output when monitor data is input.

  In this manner, a device state recognition model that recognizes the state of the device indicated by the monitor data can be automatically generated using the data acquired by the automatic monitor device. Also, accurate device control is possible based on the generated device state recognition model.

  (9) The method of the present invention is a method for automatically learning the state of a remotely controllable device, the step of transmitting a control command for controlling the target device to a specific state, and at least the control command Monitoring the status of the target device after transmitting, obtaining monitor data as a result of the monitoring, a device status label indicating a specific status of the target device, and the device status label associated with the device status label Learning the state of the target device based on the acquired monitor data, and generating a device state recognition model that gives the state of the target device as an output when the monitor data is input. To do.

  As a result, the device state recognition model that recognizes the state of the device indicated by the monitor data by using the data automatically collected by controlling and monitoring the target device in a specific state can be used as sample data for manual learning. It can be generated automatically without preparation. Also, accurate device control is possible based on the generated device state recognition model.

  (10) The program of the present invention is a program for automatically learning the state of a remotely controllable device, a process for transmitting a control command for controlling a target device to a specific state, and at least the control command Monitoring the status of the target device after transmitting the device, obtaining monitor data as a result of the monitoring, a device status label indicating a specific status of the target device, and the device status label associated with the device status label Learning the state of the target device based on the acquired monitor data, and causing the computer to execute a process of generating a device state recognition model that gives the state of the target device as an output when the monitor data is input It is characterized by.

  As a result, the device state recognition model that recognizes the state of the device indicated by the monitor data by using the data automatically collected by controlling and monitoring the target device in a specific state can be used as sample data for manual learning. It can be generated automatically without preparation. Also, accurate device control is possible based on the generated device state recognition model.

  According to the present invention, a device state recognition model for recognizing the state of the device indicated by the monitor data can be manually learned by using the data automatically collected by controlling and monitoring the target device in a specific state. It can be generated automatically without preparing sample data. Also, accurate device control is possible based on the generated device state recognition model.

It is a schematic diagram which shows the usage example of the automatic learning apparatus which concerns on 1st Embodiment. It is a block diagram which shows the structure of the automatic learning apparatus which concerns on 1st Embodiment. It is a flowchart which shows operation | movement of the automatic learning apparatus until monitor data is acquired. It is a flowchart which shows the operation | movement of the position extraction process and apparatus state recognition model production | generation of an automatic learning apparatus after acquiring monitor data. It is a schematic diagram which shows the outline | summary of a position extraction process. It is a flowchart which shows the operation | movement of evaluation of an apparatus state recognition model. It is a block diagram which shows the structure of the automatic learning system which concerns on 2nd Embodiment.

  Next, embodiments of the present invention will be described with reference to the drawings. In order to facilitate understanding of the description, the same reference numerals are given to the same components in the respective drawings, and duplicate descriptions are omitted.

[First Embodiment]
(Usage example of automatic learning device)
FIG. 1 is a schematic diagram illustrating a usage example of the automatic learning device 10. In this usage example, there are a plurality of devices that are remotely controlled by infrared rays, wireless, or the like. In addition, there is an automatic learning device 10 that can control and monitor these devices by infrared rays, wireless, or the like. As this automatic learning apparatus 10, a smart phone, a robot, etc. can be used.

  In the automatic learning apparatus 10, information on the presence of the device and information on the operation means is given in advance, but information on the location of the device and the state when the device is controlled is not given in advance. Information such as home appliance information and infrared signals for its operation can be obtained relatively easily. The automatic learning apparatus 10 may acquire device information by manual input, or may automatically acquire information on the network or the like based on a device ID (described later). Under such a premise, the automatic learning device 10 identifies a device to be learned (target device) and learns the state of the device.

  The device to be learned is preferably a device that can be controlled by command transmission such as infrared rays, but may be an IoT device capable of bidirectional communication. The automatic learning device 10 can grasp position information as described later. Therefore, even an IoT device capable of two-way communication has an advantage of using the automatic learning device 10 to acquire its position.

(Configuration of automatic learning device)
FIG. 2 is a block diagram illustrating an example of the configuration of the automatic learning device 10. The automatic learning apparatus 10 includes a device database (DB) 12, an automatic learning management unit 14, a control management unit 16, a command transmission unit 18, a monitor unit 20, a monitor data database (DB) 22, a position extraction unit 24, a learning unit 26, The model evaluation unit 28 is configured.

  The device DB 12 is a DB that holds information on a given device. For example, the following information is stored in the device DB 12. The information includes a “device ID” that uniquely identifies the device, a “device type” that indicates the type of device such as an air conditioner, a “ON / OFF state of power”, a “device state” that indicates a change in temperature, volume, etc. "State", "control command" for controlling the target device to a specific state by an infrared signal, etc., "device state recognition model" which is data for identifying the state of the device by image, for spatially identifying the control target Information, such as “target device position”. In the initial state, “device ID”, “device type”, “apparatus status”, and “control command” are given and held. Information indicating a specific state of the target device is referred to as a device state label. In the following example, information on one or more devices including a specific “device state” corresponds to the device state label.

  The automatic learning management unit 14 confirms the device DB 12, grasps a device whose identification or state has not yet been learned among existing devices, and starts automatic learning for the device. In addition, although it is preferable to start learning automatically, you may start automatic learning with a manual opportunity. The automatic learning management unit 14 notifies the control management unit 16 of what control is performed for which device to learn. The automatic learning management unit 14 may include a model evaluation unit 28 that evaluates the device state recognition model. The model evaluation unit 28 will be described later.

  The control management unit 16 performs the instructed control on the target device and performs control to monitor the change in the state. The control management unit 16 instructs the command transmission unit 18 to transmit a control command, and instructs the monitor unit 20 to start and end monitoring. The control management unit 16 preferably repeats this series of control a plurality of times so that learning about the target device notified from the automatic learning management unit 14 can be sufficiently achieved. Then, the image data (monitor data) acquired by the monitor unit 20 is associated with the device information (device state label) and stored in the monitor data DB 22. The device information is information such as a device ID, a device type, and a device state.

  The command transmission unit 18 has a function of transmitting a control command to the target device by infrared rays, wireless, or the like, and includes hardware for that purpose. The control command is, for example, a command to turn on or turn off the target device.

  The monitor unit 20 is a camera, for example, and has a function of monitoring the state of the device. It should be noted that the camera is not fixed to a specific device, but is installed at a position where image data of the entire space such as in the house can be acquired, or the direction of the lens so that the image data of the entire space can be acquired by changing the orientation. It is preferable to install it so that movement is possible. The monitor unit 20 captures an image in response to an instruction from the control management unit 16. Then, the acquired monitor data is passed to the control management unit 16. The image shooting may be continuous shooting of a plurality of images or moving image shooting. Moreover, you may image | photograph with an infrared camera etc. The monitor unit is preferably a camera as described above, but may be a video, a thermography, a microphone, or a smart meter.

  The monitor data DB 22 accumulates information about the target device and image data associated therewith. The accumulated information is used for extracting the position of the target device, learning the state of the target device, and generating a device state recognition model.

  The position extraction unit 24 extracts the position of the device in the image based on the change in the state of the target device. The position extraction unit 24 uses the monitor data stored in the monitor data DB 22 to extract the position of the device in the image. Specific position extraction processing will be described later. The extracted position is held in the device DB 12 as information for identifying the device to be controlled. The position information is preferably a position on the image, but may be a position on an actual space calculated from the image.

  The learning unit 26 has a function of learning the device state and generating a device state recognition model. Applying image recognition technology using deep learning technology, using device status labels (for example, “ON” status of a device called “light”) that represents one of the possible states of the device and image data To learn the status of the device. As a deep learning technique, Fast R-CNN can be used.

  The learning unit 26 generates a device state recognition model using the result of deep learning, and holds the device state recognition model in the device DB 12. The device state recognition model is specified by data used in a program that gives the state of the target device as an output when monitor data is input. The device state recognition model functions by causing the program that gives the state of the target device as an output when the monitor data is input to read data specifying the device state recognition model obtained by deep learning. Then, by inputting monitor data into this program, the status of the target device is output. If the device state recognition model to be read is different, the output may be different even if the same monitor data is input. In the case of a configuration including the position extraction unit 24, the position of the device in the image data can be specified using the extracted position information, and the state of the device can be learned.

  The model evaluation unit 28 evaluates the device state recognition model. The model evaluation unit 28 uses part of the acquired monitor data as test data, and evaluates whether or not the program can correctly recognize the test data when the program specifies the device state recognition model. The device state recognition model is repeatedly generated until the accuracy rate obtained as a result of the evaluation reaches a predetermined value. This is preferable because the accuracy of the device state recognition model is improved.

(Operation of automatic learning device)
Next, the operation of the automatic learning device 10 will be described. FIG. 3 is a flowchart showing an operation until monitor data is acquired. FIG. 4 is a flowchart showing operations of position extraction processing and device state recognition model generation after obtaining monitor data. As an example, an operation for learning the ON / OFF state of the indoor lamp will be described.

  First, the automatic learning management unit 14 selects a learning target device in response to an opportunity given by an operator or a periodic timer (step S1). When selecting a learning target device, the automatic learning management unit 14 makes an inquiry to the device DB 12 to create a device state recognition model or extract a target device position, or to select a low-precision learning target device. Extract and select the device as a learning target.

  The automatic learning management unit 14 inquires and extracts the information that can be taken by the selected device and information on the control command for that state (step S2). The information of the device to be extracted is information including “device ID”, “device state”, and “control command”. For example, in the case of a device that can only be turned ON / OFF with a ceiling light, two sets of information, ON or OFF, may be extracted as “device state” information.

The automatic learning management unit 14 passes the extracted device information to the control management unit 16, and starts a learning phase (step S3). The control management unit 16 selects the device x and the state s from the delivered device set (xεX) and the state set associated with x (sεS _x ) (step S4). This may be random or ordered. For example, “device: ceiling light, state: ON” is selected.

  The control management unit 16 instructs the monitor unit 20 to start monitoring (step S5). At this timing, control is not started for the device. The monitor unit 20 starts monitoring according to the instruction (step S6). Specifically, an image is taken by a camera as a monitor. In order to increase the learning amount, still image shooting or moving image shooting may be performed continuously for a certain period of time.

  The control management unit 16 instructs the command transmission unit 18 to send a control command to the device x to set the state s (step S7). Considering that the control signal is not transmitted due to interference or the like, it may be transmitted a plurality of times. The command transmission unit 18 transmits the instructed control command (step S8). For example, the control management unit 16 instructs the command transmission unit 18 to send an infrared signal control command to turn on the ceiling light. Then, the command transmitter 18 transmits an infrared signal control command associated with ON to the ceiling light.

  The control management unit 16 starts monitoring, and instructs to stop monitoring after a sufficient time for device control (for example, after 10 seconds) (step S9). The predetermined expected control time may be extended to 30 seconds by adding a margin time in order to secure a margin. The monitor unit 20 stops monitoring in response to the instruction (step S10).

  The monitor unit 20 transmits the captured and recorded data to the control management unit 16 after the monitoring is stopped (step S11). The control management unit 16 associates the instructed device x and state s with the acquired monitor data (step S12). For example, the control management unit 16 associates the transmitted monitor data with “device: ceiling light, state: ON”.

  It is determined whether or not all the states are sufficiently repeated including whether or not there are other states instructed to the device x (step S13). Whether or not it is sufficient can be determined, for example, based on whether or not monitoring has reached a predetermined number of times for one state.

  If all the states have not been sufficiently repeated, such as other states instructed to the device x, the process returns to step S4 again, and the same operation is repeated in the other states. If there is an instruction from a device other than device x, the process returns to step S4 again, and the same operation is repeated in the device other than device x. Note that it is necessary to be able to take at least two device states as a premise of learning so that monitoring in the same state is not repeated continuously.

  When the repetition of the operation is completed for all the instructed devices and states, the result is stored in the monitor data DB 22 (step S14). In this example, every time device state control is monitored, information of “device: ceiling light, state: ON, monitor data: multiple images” is stored in the monitor data DB 22.

  When the data collection by the control management unit 16 is completed, the position extraction unit 24 next extracts the position of the device. For extraction, first, monitor data in a different state is extracted from the monitor data DB 22 by the same device (step T1). Then, position extraction processing is performed using the extracted monitor data (step T2). Details of the position extraction process will be described later.

  The learning unit 26 uses the device x and its state s in the image as labels, extracts the device image in the image based on the device position information, and performs device image learning processing (step T2). This learning process uses a deep learning technique. For example, Imagenet can be used as an image recognition program for identifying the state from an image.

  When performing the learning process, it is preferable to perform the following pre-processes (A) and (B) in order to remove the monitor data when the device control by the control command fails. (A) First, for each monitor result in each state, data after a fixed time has elapsed from the start of monitoring (for example, after 5 seconds) is extracted as a state change completed. Further, the similarity between the monitor data may be obtained along a time series, and the monitor data after the time when the largest change is generated may be extracted as the monitor data whose state change is completed.

  (B) The similarity between each monitor data is calculated with respect to the monitor data filtered by said (A). In this case, N (N−1) / 2 similarities are calculated for N monitor data. Among them, a set of N constant rates (for example, 75%) is created, the average value of the similarity of the monitor data in the set is calculated, and the set of monitor data having the highest average value is learned. Extract as target. As a result, monitor data in a state different from the intention mixed exceptionally can be removed. For example, when there is monitor data A, B, C, and D, if the constant rate is 75%, four sets of ABC, BCD, ABD, and ACD are formed. At this time, the similarity between A and B, B and C, and A and C can be calculated in the set ABC, and the average value thereof can be obtained. Similarly, the average value of the similarity is calculated for other sets, and the monitor data of the set having the highest average similarity is extracted as a learning target. The learning unit 26 stores the learned result in the device DB as a device state recognition model (step T4).

(Position extraction)
FIG. 5 is a schematic diagram showing an outline of the position extraction process. The monitor data DB 22 stores a plurality of monitor data in different states for each device. The position of each device can be extracted by taking the difference between the monitor data in different states for each device. For example, monitor data with the ceiling light turned on and monitor data with the ceiling light turned off can be used. The brightness of the ceiling light changes between when the ceiling light is on and when it is off, and the position of the ceiling light as the target device can be extracted from the difference. The periphery including the change range is learned as the position of the device and stored in the device DB.

  Further, for example, by controlling the air conditioner from OFF to ON and taking the difference between the image data in each state, it is possible to grasp that the position where the luminance has changed is the power lamp. In this case, the area including the position where the luminance has changed is handled as the position of the device. In addition, when the shape of the air blowing port is changed, an area including the position where the shape is changed can be handled as the position of the device.

  When there are three or more device states, a change position (a most changed position) common to a plurality of device states may be extracted. Note that this position does not need to be strictly specified by surrounding only the device, and it is sufficient if the uniqueness of being in this coordinate can be ensured in order to uniquely identify the device.

(Evaluation of equipment state recognition model)
FIG. 6 is a flowchart showing an operation of evaluating the device state recognition model. The model evaluation unit 28 extracts a device state recognition model from the device DB 12 (step U1). Then, the device state recognition model is evaluated (step U2). The model evaluation unit 28 uses part of the acquired monitor data as test data, and evaluates whether or not the program can correctly recognize the test data when the program specifies the device state recognition model. Do. When the accuracy rate obtained as a result of the evaluation does not reach the predetermined value, the process returns to step S5, and monitor data acquisition and learning and a device state recognition model are generated (step U3). When the accuracy rate obtained as a result of the evaluation reaches a predetermined value, the fact is stored in the device DB 12 and the process is terminated (step U4).

(Modified example of monitor data)
As the monitor data, it is preferable to use the image data photographed by the camera as described above, but the sound collected by the microphone or the power consumption value collected by the smart meter may be used. In this case, position information is basically not used. When the monitor data is sound, the operation sound and operation sound of the device, the volume of the device having a speaker, and the like can be used for learning. When the monitor data is a power consumption value, a waveform or power consumption value specific to the device can be used for learning. The monitor unit 20 may collect a plurality of different types of monitor data. At that time, the learning unit 26 may separately learn different types of monitor data, or may learn data combining them. May be.

[Second Embodiment]
In the above-described embodiment, the automatic learning device 10 is configured to be able to perform all the position extraction and learning processing in its own device, but the command transmission unit 18 and the monitor unit 20 need to be in the same place as the target device. For example, some functions may be provided on the cloud.

  FIG. 7 is a block diagram illustrating an example of the configuration of the automatic learning system 50. The automatic learning system 50 includes an automatic monitoring device 100 and a learning device 200. The automatic monitoring device 100 automatically monitors the state of a remotely controllable device. The learning device 200 can communicate with the automatic monitor device 100, and uses the monitor data acquired from the automatic monitor device 100 to perform position extraction of the target device and learning processing.

(Configuration of automatic monitoring device)
The automatic monitoring device 100 includes a control management unit 16, a command transmission unit 18, a monitor unit 20, and a monitor side communication unit 130. The functions of the control management unit 16, the command transmission unit 18, and the monitor unit 20 are the same as those in the first embodiment, and will be omitted.

  The monitor side communication unit 130 communicates with a learning side communication unit 230 provided in the learning device 200. The monitor-side communication unit 130 receives necessary data stored in the device DB 12 and the monitor data DB 22 provided in the learning device 200 and transmits monitor data associated with the device state label by the control management unit 16. .

(Configuration of learning device)
The learning device 200 includes a device DB 12, an automatic learning management unit 14, a monitor data DB 22, a position extraction unit 24, a learning unit 26, a model evaluation unit 28, and a learning side communication unit 230. The functions of the device DB 12, the automatic learning management unit 14, the monitor data DB 22, the position extraction unit 24, the learning unit 26, and the model evaluation unit 28 are the same as those in the first embodiment, and are omitted.

  The learning side communication unit 230 communicates with the monitor side communication unit 130 provided in the automatic monitoring apparatus 100. The learning-side communication unit 230 transmits necessary data stored in the device DB 12 and the monitor data DB 22, receives monitor data associated with the device state label by the control management unit 16, and stores the monitor data in the monitor data DB 22. .

(Operation of automatic learning system)
The operation of the automatic learning system 50 is the same as the operation of the automatic learning apparatus 10 of the first embodiment except that necessary data is exchanged between the monitor side communication unit 130 and the learning side communication unit 230.

  In this manner, by providing the learning unit 226 on the cloud, it is possible to cope on the cloud even if the amount of calculation necessary for learning increases. In addition, a device state recognition model generated from monitor data acquired by another automatic monitor device 100 can be used, or monitor data acquired by a plurality of automatic monitor devices 100 can be combined into learning data.

10 Automatic learning device 12 Equipment DB
14 Automatic learning management unit 16 Control management unit 18 Command transmission unit 20 Monitor unit 22 Monitor data DB
24 position extraction unit 26 learning unit 28 model evaluation unit 50 automatic learning system 100 automatic monitor device 130 monitor side communication unit 200 learning device 230 learning side communication unit

Claims (10)

An automatic learning device that automatically learns the state of a remotely controllable device,
A command transmission unit that transmits a control command for controlling the target device to a specific state;
A monitor unit that monitors at least the state of the target device after transmitting the control command, and obtains monitor data as a result of the monitoring;
The device status label indicating the specific status of the target device and the status of the target device is learned based on the acquired monitor data associated with the device status label, and the monitor data is input as the target. An automatic learning apparatus comprising: a learning unit that generates a device state recognition model that gives a device state as an output. A model evaluation unit for evaluating the device state recognition model;
The model evaluation unit uses part of the acquired monitor data as test data, evaluates whether the device state recognition model can correctly recognize the test data, and obtains a correct answer rate obtained as a result of the evaluation. The automatic learning apparatus according to claim 1, wherein the device state recognition model is repeatedly generated until a predetermined value is reached.   The learning unit selects monitor data with high similarity from the monitor data in the specific state as representative monitor data, and generates the device state recognition model with the representative monitor data as a learning target. The automatic learning apparatus according to claim 1 or 2.   The automatic learning apparatus according to claim 1, wherein the monitor unit includes a camera and acquires image data as the acquired monitor data.   The automatic learning device according to claim 4, further comprising a position extracting unit that extracts a position of a device from a difference between monitor data having different device status labels associated with each other in the acquired monitor data.   The automatic learning device according to any one of claims 1 to 3, wherein the monitor unit includes a microphone and acquires voice data as the acquired monitor data. An automatic monitoring device that automatically monitors the state of a remotely controllable device,
A command transmission unit that transmits a control command for controlling the target device to a specific state;
A monitor unit that monitors at least the state of the target device after transmitting the control command and obtains monitor data as a result of the monitoring, and
The device status label indicating the specific status of the target device and the acquired monitor data associated with the device status label are used for learning of the status of the target device in a separate automatic learning device. An automatic monitor device characterized in that a device state recognition model is generated that gives the state of a target device as an output when monitor data is input. An automatic learning system that automatically learns the status of remotely controllable devices,
An automatic monitoring device according to claim 7;
A data acquisition unit that acquires a device status label indicating a specific state of the acquired target device and monitor data associated with the device status label, and the target based on the acquired device status label and monitor data An automatic learning system comprising: a learning device that learns a device state and has a learning unit that generates a device state recognition model that outputs the state of the target device as an output when monitor data is input . A method for automatically learning the state of a remotely controllable device,
Transmitting a control command for controlling the target device to a specific state;
Monitoring at least the state of the target device after transmitting the control command, and obtaining monitor data as a result of the monitoring;
The device status label indicating the specific status of the target device and the status of the target device is learned based on the acquired monitor data associated with the device status label, and the monitor data is input as the target. Generating a device state recognition model that provides the device state as an output. A program that automatically learns the status of remotely controllable devices,
A process of sending a control command to control the target device to a specific state;
A process of monitoring at least the state of the target device after transmitting the control command, and obtaining monitor data as a result of the monitoring;
The device status label indicating the specific status of the target device and the status of the target device is learned based on the acquired monitor data associated with the device status label, and the monitor data is input as the target. A program that causes a computer to execute processing for generating a device state recognition model that gives a device state as an output.

Images