KR102029751B1 - Sensor device able to monitor external environment based on sound or image and environment monitoring system comprsing the sensor device - Google Patents

Abstract

The present invention provides a first sensor unit including an image sensor and / or a sound sensor, a communication unit for transmitting and receiving data through a wireless network, a storage unit for storing one or more programs for processing sensing data from the first sensor unit, and one or more programs. The present invention relates to a sensor device and an environmental monitoring system including a control unit which executes a program to process sensing data from a first sensor unit to generate result data, and transmit the generated result data through a communication unit.

Description

Sensor device capable of monitoring the external environment based on sound or video and environmental monitoring system including the same

The present invention relates to a sensor device capable of monitoring an external environment based on sound or an image, and an environmental monitoring system including the same, and specifically, recognizes an external environment by processing data sensed from a sound sensor and / or an image sensor. And a sensor device capable of monitoring an external environment based on sound or video and configured to transmit a result value indicating a recognized state through an IoT network, and an environmental monitoring system including the same.

Efforts have been made to monitor or measure the living environment using the Internet of Things (IoT) technology to detect risks in advance or to provide a safe environment.

To this end, IoT devices are equipped with various sensors to monitor various external environments. IoT devices are configured to monitor the external environment using various sensors such as air quality measurement sensors (PM2.5, VOCs), radiation sensors, and electromagnetic sensors.

In addition, developers are interested in the use of image sensors and sound sensors corresponding to vision and hearing, which play the most important sensing role among the human senses in IoT devices.

In general, video and sound data is sent to a server (cloud server) as it is collected, and the server processes the collected data. The cloud server may provide various services related to video and sound by processing the collected data due to its high computing power.

The IoT device may transmit and receive various data with the cloud server through the IoT network. The IoT network may be, for example, a long range (LoRa) network or a narrowband-Internet of things (NB-IoT) network. Unlike conventional wired LAN, Wi-Fi, and LTE networks, the existing IoT network has a low communication speed. For example, on LoRa or NB-IoT networks, IoT devices have communication speeds ranging from a few bps to a few Kbps.

This IoT communication speed is far short of processing video and sound data. In addition, the video and sound data may include information sensitive to privacy, and may cause various problems with respect to privacy when the video and sound data itself is transmitted to the cloud server.

As such, there is a need for a sensor device capable of monitoring an external environment based on sound or video and an environmental monitoring system including the same, which can solve various problems caused in processing of video and sound data through an IoT network. .

The present invention has been made to solve the above-described problems, the external environment based on the sound or video, to monitor the external environment based on the image or sound sensing and to transmit the monitored result through the low-speed IoT network An object of the present invention is to provide a sensor device capable of monitoring and an environmental monitoring system including the same.

In addition, the present invention is configured to recognize only the designated object from the image or sound input using the deep-learning program and to calculate the result data based on the recognized object to prevent exposure of personal information by the image or sound data It is an object of the present invention to provide a sensor device capable of monitoring an external environment based on sound or image, and an environmental monitoring system including the same.

In addition, the present invention is based on the external sound based on the sound or video to update the embedded program using the sensing data recorded in each sensor device and to redistribute the updated embedded program to the sensor device through a low-speed IoT network An object thereof is to provide a sensor device capable of monitoring the environment and an environmental monitoring system including the same.

In addition, the present invention, in conjunction with other environmental sensors in conjunction with the event of the environmental sensor to sense the image or sound data and the object specified in the event and to calculate the resulting data accordingly, so that the sound or An object of the present invention is to provide a sensor device capable of monitoring an external environment based on an image and an environmental monitoring system including the same.

The technical problems to be achieved in the present invention are not limited to the technical problems mentioned above, and other technical problems not mentioned above will be clearly understood by those skilled in the art from the following description. Could be.

According to an aspect of the present invention, a sensor device includes an image sensor, a sound sensor, or a first sensor unit including an image sensor and a sound sensor, a communication unit for transmitting and receiving data through a wireless network, and processing sensing data from the first sensor unit. A storage unit for storing one or more programs and a control unit for performing the one or more programs to process the sensing data from the first sensor unit to generate the result data and to send the generated result data through the communication unit.

In the sensor device described above, the wireless network is a LoRa network or an NB-IoT network, and the one or more programs include a deep-learning program capable of recognizing a set object and generating result data in accordance with the recognition of the object.

The sensor device may further include a second sensor unit including one or more environmental sensors different from the image sensor and the sound sensor, wherein the control unit recognizes an external event and recognizes an external event in sensing data from the second sensor unit. In this case, the first sensor unit is driven to recognize a target mapped to an external event and generate result data according to the recognition of the target.

The sensor device may further include a power supply unit configured to supply power corresponding to a voltage level specified in the first sensor unit, the second sensor unit, the communication unit, the storage unit, and the control unit, and to control the power supply of the first sensor unit. The controller which cuts off the power supply to the first sensor unit through the power supply unit supplies power corresponding to the voltage level specified by the first sensor unit through the power supply unit when the external event is recognized through the second sensor unit. The first sensor unit is driven to recognize a target mapped to an external event in sensing data from the first sensor unit, and generate result data according to the recognition of the target.

In the above sensor device, the first sensor unit includes an image sensor and a sound sensor, and the sensor device supplies power corresponding to a voltage level specified to the image sensor and the sound sensor and controls power supply of the image sensor. The apparatus further includes a supply unit, and when the controller recognizes an external event specified in the sound data from the sound sensor, the controller supplies power to the image sensor through the power supply unit and processes the image data from the image sensor to recognize the designated object and designate the designated object. Generates the result data according to the recognition of the transmitted through the communication unit.

In the above sensor device, the one or more programs further comprise an update program for updating the deep-learning program, wherein the controller for performing the update program is collected by the plurality of sensor devices via the LoRa network or the NB-IoT network. Further learning by sensing data receives portions of the updated deep-learning program at different times and combines the portions to replace the deep-learning program of the storage.

In the sensor device, the storage unit includes a detachable memory card, and the controller recognizing the external event stores the sensing data received through the image sensor and the sound sensor included in the first sensor unit according to the recognition of the external event. The sensing data stored in the card and stored in the memory card is used for updating a deep learning program included in one or more programs through further learning by the cloud server.

An environmental monitoring system according to an aspect of the present invention includes the first sensor device and the second sensor device described above.

The above-described environmental monitoring system further includes a cloud server that receives the result data from the first sensor device and the second sensor device wirelessly through the IoT network and performs corresponding processing for each sensor device.

In the above-described environmental monitoring system, the cloud server uses a deep learning program performed in the first sensor device and the second sensor device by using offline sensing data collected from the first sensor device and the second sensor device as additional learning data. And update the partial files of the updated deep-learning program to the first sensor device and the second sensor device, wherein the first sensor device receives the partial file received at the first sensor device and the partial file received at the second sensor device. The updated deep learning program is configured to replace the existing deep learning program stored in the storage unit with the updated deep learning program.

The sensor device capable of monitoring the external environment based on the sound or the image according to the present invention as described above and the environmental monitoring system including the same monitor the external environment based on the image or sound sensing and monitor the monitored result at a low speed IoT network. It is effective to send through.

In addition, the sensor device capable of monitoring the external environment based on the sound or the image according to the present invention as described above, and the environmental monitoring system including the same recognize only the designated object from the image or sound input using the deep-learning program It is effective to prevent personal information exposure by video or sound data by configuring the result data based on the recognized object.

In addition, the sensor device capable of monitoring the external environment based on the sound or image according to the present invention as described above and the environmental monitoring system including the same update the embedded program using the sensing data recorded in each sensor device It has the effect of redistributing updated embedded programs to sensor devices via low-speed IoT networks.

In addition, the sensor device capable of monitoring the external environment based on the sound or the image according to the present invention as described above, and the environmental monitoring system including the same, in conjunction with other environmental sensors in conjunction with the event of the environmental sensor image or sound data It has the effect of recognizing the targets specified in the sensing and events, and calculating the result data and sending them to the IoT network.

The effects obtainable in the present invention are not limited to the above-mentioned effects, and other effects not mentioned above may be clearly understood by those skilled in the art from the following description. will be.

1 is a view showing an example of the connection configuration between the devices configured in the environmental monitoring system according to the present invention.
2 shows an exemplary block diagram of a sensor device.
3 illustrates an exemplary control flow that may utilize a sensor device in accordance with the present invention to monitor an external environment and update a program accordingly.

The above objects, features, and advantages will become more apparent from the detailed description given hereinafter with reference to the accompanying drawings, and accordingly, those skilled in the art to which the present invention pertains may share the technical idea of the present invention. It will be easy to implement. In addition, in describing the present invention, when it is determined that the detailed description of the known technology related to the present invention may unnecessarily obscure the gist of the present invention, the detailed description thereof will be omitted. Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings.

1 is a view showing an example of the connection configuration between the devices configured in the environmental monitoring system according to the present invention.

The environmental monitoring system of FIG. 1 is configured to monitor an external environment based on sound and / or video.

Referring to FIG. 1, an environmental monitoring system includes at least one sensor device 100, a wireless network 200, a wireless AP 300, a broadband network 400, a cloud server 500, and an external device. Server 600.

The sensor device 100 is a device capable of monitoring an environment outside the sensor device 100. The sensor device 100 has at least a sound sensor and / or an image sensor to capture (recognize) the sound data and / or the image data via the sound sensor and / or the image sensor and to designate the sound data and / or the image data ( It is configured to recognize the set) event or the designated (set) target (target) and to transmit the result data calculated according to the recognized event or target through the wireless network 200.

The sensor device 100 includes a program for calculating the result data, and is configured to recognize a specific target or event from sound data and / or image data through the program. This program may for example be an artificial intelligence program and for example may be a deep-learning program.

The sensor device 100 may be configured to be fixedly installed or movable at a specific position. The sensor device 100 may be an IoT device that may be connected to an IoT network and may sense various external environments. The sensor device 100 will be described in detail later with reference to FIG. 2.

The wireless network 200 is a network for wirelessly transmitting and receiving data between various fixed or movable sensor devices 100. The wireless network 200 may connect the sensor devices 100 within a certain area to each other. The wireless network 200 may be a LoRa network or an NB-IoT network capable of wireless communication within an area range of several hundred meters or several kilometers and used as an IoT network. In addition, the wireless network 200 may be a Zigbee network that can be utilized for the sensor network or Wi-Fi.

The wireless AP 300 (Access Point) is a device for connecting the wireless network 200 and the broadband network 400. The wireless AP 300 may transmit the result data received from the sensor device 100 through the wireless network 200 to the cloud server 500 through the broadband network 400. The wireless AP 300 may transmit various data from the cloud server 500 to one or a plurality of sensor devices 100 on the connected wireless network 200.

Here, each sensor device 100 connected to the wireless network 200 may have the same program and configuration to monitor the same designated external environment. For example, the sensor device 100 may include a deep learning program that recognizes one or more specific events or specific objects from an image or sound through deep learning, and may include the same environmental sensor.

The broadband network 400 is a network capable of transmitting and receiving data in a wider area than the single communication area covered by the wireless network 200. Broadband network 400 is composed of, for example, a mobile communication network, an internet network, or a combination thereof provided by a mobile service provider.

Through the broadband network 400, a server or the like may transmit various data to the sensor device 100 through the wireless AP 300 and receive the data from the sensor device 100.

The cloud server 500 is connected to the plurality of sensor devices 100 through the broadband network 400 and is configured to process various data from each of the plurality of sensor devices 100 and to output a processing result.

For example, the cloud server 500 receives result data from each of the sensor devices 100 wirelessly through an IoT network, which is an example of the wireless network 200, and uses the result data to transmit to the respective sensor device 100. Perform the corresponding process.

For example, the cloud server 500 may use result data including whether the person is recognized from the sensor device 100, whether the vehicle is recognized, the number of people recognized, the number of vehicles recognized, and the like (and recognized external events). A specific event according to the purpose of the sensor device 100 is recognized and corresponding processing (for example, fire department contact, police station contact, vehicle control, etc.) is performed. Correspondence processing includes, for example, the cloud server 500 establishing a communication channel to a designated external server 600, including an event recognized by the external server 600 and location information of the sensor device 100, and requesting a specific action. It can be configured by sending data.

In addition, the cloud server 500 updates a program performed in the sensor device 100 connected to the wireless network 200 using data collected from the sensor devices 100 and updates the updated program to the wireless network 200. Can be transmitted to the respective sensor devices 100 via.

For example, the cloud server 500 recognizes sensing data of a removable memory card provided in the sensor device 100 as preparation and input by an administrator or the like, and senses sensing data of the various sensor devices 100 and further results data, Using self-learning as additional learning data to update the deep learning program embedded in the sensor device 100.

The cloud server 500 transmits the updated deep-learning program to the various sensor devices 100 performing the deep-learning program through the wireless network 200. Given that the transmission speed of the IoT network is low, the cloud server 500 divides the updated deep learning program into several parts (multiple patch files) and transmits the divided parts to the sensor devices 100 at different times. .

The cloud server 500 transmits different parts of the updated deep-learning programs to each sensor device 100 connected to a group of the same IoT network for efficiency of program transmission, and each sensor device 100 Receives different parts from the other sensor device 100 connected to the IoT network and may configure the deep-learning program updated by the cloud server 500 using the parts different from the received part.

The sensor device 100 combines the parts (patch files) received from the various sensor devices 100 in the same wireless network 200 to form the entire deep-learning program and store the existing data stored in the storage unit 107. The deep learning program can be replaced with an updated deep learning program.

The external server 600 is connected to the cloud server 500 through the broadband network 400 to receive a specific action request and process the action. The external server 600 may be, for example, a server installed in a specific institution such as a police department, a fire department, a traffic situation room, or the like.

2 shows an exemplary block diagram of a sensor device 100.

According to FIG. 2, the sensor device 100 includes a first sensor unit 101, a second sensor unit 103, a communication unit 105, a storage unit 107, an output unit 109, a power supply unit 111, and The control unit 113 is included. According to a design example, the sensor device 100 may omit certain components of FIG. 2 and may further include other components not shown.

Referring to the sensor device 100 in detail with reference to FIG. 2, the first sensor unit 101 includes an image sensor, a sound sensor or an image sensor, and / or an image image from an included image sensor and / or a sound sensor. Or recognizing (capturing) sound and outputting recognized image data and / or sound data.

The image sensor may be, for example, a camera sensor, a CCD sensor, a CMOS sensor, or the like. The sound sensor includes a microphone that converts the vibration signal into an electrical signal. The first sensor unit 101 may further include logic (chipset) for converting an image signal output from an image sensor into a digital signal and an audio ADC (or CODEC) for converting an electrical signal from a micro signal into a digital signal. have.

The first sensor unit 101 may receive power of a specified voltage level from the power supply unit 111 to output image data and / or sound data, and the capturing of the image or sound data is stopped when the power supply is cut off. .

The second sensor unit 103 senses specific environmental data specified outside the sensor device 100. The second sensor unit 103 may sense various environmental data, including one or more environmental sensors. Environmental sensors may include, for example, infrared sensors, air quality sensors, radiation sensors, electromagnetic wave sensors, fire detection sensors, and the like.

Environmental sensors can monitor the occurrence of specific events and monitor a variety of events, such as unauthorized intrusions from outside, environmental pollution, fire detection, and the presence of people.

The environmental sensor may have a configuration different from that of the image sensor and the sound sensor of the first sensor unit 101 to sense specific environmental data and output the sensed data.

The communication unit 105 transmits and receives data through the wireless network 200. The communication unit 105 may include an antenna and a communication chipset (module) for applying a wireless communication technology of the wireless network 200 to recognize a wireless packet from the air or to transmit a wireless packet to the air.

For example, the communication unit 105 may include a wireless antenna and a chipset for using LoRa network (communication) technology to transmit and receive various wireless packets according to the LoRa standard. Alternatively, the communication unit 105 may include a wireless antenna and a chipset for using the NB-IoT network (communication) technology to transmit and receive various wireless packets according to the NB-IoT standard.

The storage unit 107 stores various data and programs, including a volatile memory, a nonvolatile memory, and / or a removable memory card.

For example, the storage unit 107 may include a control program for collectively controlling the sensor device 100 in a nonvolatile memory, a detection program for recognizing a specific event from sensing data through the second sensor unit 103, and a first program. A processing program for processing sensing data through the sensor unit 101 and an update program for updating a program stored in the storage unit 107 are stored.

The control program controls the sensor device 100 as a whole and can control the driving and termination of other programs. The sensing program may detect a specific external event designated according to the type of the environmental sensor of the second sensor unit 103. For example, detection programs can be configured to recognize events such as unauthorized intrusions, environmental pollution abnormalities, accidents, release of toxic substances, and fires using sensing values of environmental sensors. The sensing program may be composed of a general programming technique or a deep learning program according to a design example.

The processing program is configured to process the sensing data of the first sensor unit 101 to calculate a designated result value. For example, the processing program may include a deep learning program that recognizes a set object (a person, a face of a person, a vehicle, a vehicle number, a particular insect) and generates the result data according to whether the object is recognized, the number of the recognition object, and the like. Can be.

The deep-learning program may receive image data and sound data of the first sensor unit 101 as an input and calculate a recognition result through a node (neural network) having a multilayer structure. According to the present invention, a deep learning program is configured to recognize a specific object. For example, the deep-learning program may recognize a person from the image data (additionally using sound data), recognize a particular insect, a specific animal, or recognize a vehicle.

According to the present invention, the deep-learning program can be optimized for the recognition of a specific object and can simplify the hierarchical structure of the neural network (for example, can be modified to have a middle layer node of 5 layers or less).

The update program makes it possible to update one or more programs stored in the storage unit 107. The update program may be driven by a control program and may update a specific program through the over-the-air technique. For example, the update program receives the updated deep-learning program of the processing program from the wireless network 200 (for example, from the cloud server 500) via the communication unit 105 and the existing dip of the storage unit 107. Can replace running programs

Each program stored in the storage unit 107 is configured to interoperate with each other through various known programming techniques.

The storage unit 107 includes a removable memory card (for example, an SD card), and the memory card includes sensing data (image data and / or audio signals received through the image sensor and / or the sound sensor of the first sensor unit 101). And / or sound data), and further the resulting data.

The stored sensing data is provided to the cloud server 500 through offline. The cloud server 500 is used to update existing deep-learning programs using sensing data provided from various sensor devices 100 as learning data.

The output unit 109 includes a speaker, a buzzer, a siren, an LED, and the like to output a signal from the control unit 113. For example, the output unit 109 may receive a warning signal from the control unit 113 and output it as a sound through a speaker, a buzzer, a siren, or a light through an LED.

The power supply 111 includes a power source (for example, a lithium ion battery, a dry battery, an AC power source, etc.) and includes a first sensor unit 101, a second sensor unit 103, and a storage unit 107 from a power source. The first sensor unit 101, the second sensor unit 103, and the storage unit 107 generate one or more power sources to be used in the communication unit 105, the output unit 109, and the control unit 113. To the communication unit 105, the output unit 109, and the control unit 113.

For example, power supply 111 may include one or more voltage regulators to generate power from a power source corresponding to a specified voltage level (3.3V, 5V, 12V, etc.) for which each component is required and to generate the generated power. Can output to the component.

The power supply 111 is configured to control the power supply to a specific component. The power supply 111 may include a power switch (relay) for controlling the supply / blocking of power to control power supply to a specific component. For example, the power supply 111 includes a power switch for supplying and cutting off a voltage level specified for the first sensor unit 101 (image sensor and / or sound sensor) and a control signal from the control unit 113. In accordance with this, the first sensor unit 101 (the image sensor and / or the sound sensor) may supply or cut off power at a specified voltage level.

When the first sensor unit 101 includes both an image sensor and a sound sensor, the power supply unit 111 includes a power switch for individually controlling power of the image sensor and the sound sensor, respectively, from the control unit 113. The power supply to each sensor can be controlled according to the control signal.

The controller 113 includes one or more execution units capable of executing instructions of the program to control the sensor device 100 using one or more programs stored in the storage unit 107. The controller 113 may be referred to as or include a processor, a microcomputer, a CPU, an MPU, a central processing unit, and the like.

For example, the controller 113 may perform one or more programs of the storage unit 107 to process sensing data from the first sensor unit 101 to generate result data, and to generate the result data in the communication unit 105. Send to the cloud server 500 through.

The controller 113 may load or control a program to drive or control other programs, and monitor an external environment through a processing program (deep-learning program), a detection program, an update program, and the like. Can be updated.

In addition, the controller 113 controls the power supply 111 through a control program. For example, the control program outputs a control signal for controlling a power switch capable of supplying or interrupting power of a specified voltage level to the first sensor unit 101 (the image sensor of) and accordingly the first sensor unit 101. Drive the image sensor) and receive the sensing data (the image data).

An exemplary control flow made by the controller 113 will be described in more detail with reference to FIG. 3.

3 is a diagram illustrating an exemplary control flow that may utilize a sensor device 100 in accordance with the present invention to monitor an external environment and update a program accordingly.

According to the present invention, the environmental monitoring system can be utilized for various applications. For example, environmental monitoring systems may include external intrusion detection (such as single households or toilets), indoor environment (eg classroom) monitoring, access monitoring (such as number of visitors), traffic environment analysis, parking lot monitoring, accident monitoring, noise or air quality. It is a system that can perform monitoring, fire detection and danger detection.

The sensor device 100 of the environmental monitoring system is configured to include at least the first sensor unit 101 so as to recognize a specific object using image and / or sound data and perform a corresponding action or process.

The sensor device 100 of the environmental monitoring system includes a specific environmental sensor according to an application example and configured to recognize an event specific to the application example and the like. In addition, the sensor device 100 has a program specific to the application example, and preferably has a deep learning program optimized and used for object recognition (human, animal, insect, etc.) in the application example.

Here, the specific process of the control flow of FIG. 3 may be omitted according to an application example. For example, step S10 or S20 may be omitted depending on the application.

Referring to the control flow through FIG. 3, the sensor device 100 (control unit 113) recognizes an external event through a sensing program (S10).

For example, the sensor device 100 (control unit 113) that performs a sensing program may receive sensing data from the second sensor unit 103 and recognize the occurrence of a specified external event according to the value of the received sensing data. have. The sensor device 100 (control unit 113) recognizes a specific event (external intrusion, absence, air pollution, fire occurrence, danger detection, etc.) by comparing the environmental data received from the second sensor unit 103 with a threshold value. can do.

Alternatively, the sensor device 100 (control unit 113) that executes the sensing program or the deep learning program may receive sound data from the sound sensor of the first sensor unit 101 and recognize the occurrence of a specified external event. . The sensor device 100 (control unit 113) converts sound data into feature data through frequency conversion and compares the converted feature data with feature data corresponding to an existing event to determine whether an external event has occurred. Can be. For example, the sensor device 100 (control unit 113) can recognize the occurrence of events such as screaming and treble.

When the sensor device 100 (control unit 113) utilizes only a sound sensor for recognition of an external event, the second sensor unit 103 may be omitted from the sensor device 100. In addition, the process of S10 and the subsequent process of S20 may be omitted according to the snow scene modification.

If the sensor device 100 (control unit 113) does not recognize the external event, the process S10 is repeatedly performed before the process after S20 is performed so that the sensor device 100 (control unit 113) of the external event Recognition can be continually attempted.

The sensor device 100 (control unit 113) drives the first sensor unit 101 according to recognition of an external event through a control program (S20). For example, the sensor device 100 (control unit 113), which cuts off power to the first sensor unit 101 (image sensor), supplies power to the first sensor unit 101 (image sensor). It generates a control signal for outputting and outputs the generated control signal to the power supply 111.

According to the reception of a control signal for power supply, the power supply 111 is connected to the first sensor unit 101 (the image sensor) and a signal corresponding to the control signal received at the power switch configured to supply or cut off power. Is applied to supply the power corresponding to the specified voltage level to the first sensor unit 101 (image sensor).

As the power is supplied, the first sensor unit 101 (the image sensor) starts to drive and outputs sensing data including the image data from the image sensor and / or sound data from the sound sensor to the controller 113. .

The sensor device 100 (control unit 113) recognizes a designated object by using the received sensing data, and maps sensing data including image data and / or sound data to storage time (start time and end time). In operation S30, the memory card is stored in a removable memory card of the storage unit 107 through a control program or the like. Preferably, the sensor device 100 (control unit 113) may store the corresponding image data and sound data in the memory card when the external event is recognized.

The sensor device 100 (control unit 113) recognizes a specific object mapped to a specific external event in sensing data (image data, and also sound data) through a deep learning program driven by a control program or the like and recognizes the object. The result data is generated and the generated result data is transmitted to the cloud server 500 through the wireless network 200 (S40).

Specifically, the sensor device 100 (control unit 113) is configured to map an object to be recognized according to the occurrence of a specific external event. For example, in case of an intrusion event, air pollution event, fire occurrence, danger occurrence, etc., a person (human face) is recognized as a target (target) or a vehicle and / or a person or environment in the event of a vehicle accident. Deep-learning programs are organized to recognize specific insects, for example for protection or care. The specific object is mapped according to the environmental sensor and the event generated as described above, and the deep-learning program is configured to recognize the object.

The sensor device 100 (control unit 113) recognizes the designated object by inputting the image data (the sound data) from the image sensor through the driving of the deep-learning program, and results based on the recognition of the designated object (based on). Generate data.

For example, the sensor device 100 (control unit 113) may be a result data including whether the object is recognized, the number of objects recognized and the recognition time and the external event data further recognized through the deep-learning program and the control program It generates and sends it to the cloud server 500 through the communication unit 105.

The cloud server 500 may perform a corresponding process or action in association with the external server 600 according to the received result data. For example, when a specific external event occurs and a person or the like is recognized in the event, the cloud server 600 may cause an accident, fire, environmental pollution, or emergency by an external server 600 such as a fire department or a police agency. In addition to the event data such as data, data indicating whether there is a person or the like and the number of people may be transmitted to request that a corresponding action be performed.

In addition, the sensor device 100 (control unit 113) maps the result data to sensing data (image data and / or sound data) stored in the memory card and stores the result data in the memory card.

Meanwhile, the cloud server 500 updates the deep learning program using additional learning data (S50).

The administrator of the environmental monitoring system according to the present invention separates the memory cards of the various sensor devices 100 for monitoring the same environment from the sensor device 100 and stores the sensing data (and corresponding result data) of the memory card in the deep-learning program. It can be provided to the cloud server 500 as learning data used when updating.

The cloud server 500 updates the existing deep learning program using sensing data (and corresponding result data) from various sensor devices 100 that can be used to update the deep learning program as learning data.

Thereafter, the cloud server 500 distributes the updated deep-learning program to various sensor devices 100 according to the OTA standard through the wireless network 200 (eg, LoRa network or NB-IoT network) (S60). do.

In view of the speed of the wireless network 200 and the limitations of the opportunities of connection of each sensor device 100 to the wireless network 200, the cloud server 500 may update the updated deep-learning program in several parts (patch files). And the separated parts can be received at different times in the sensor device 100.

For example, the cloud server 500 may transmit several portions to the single sensor device 100 sequentially at different times over the wireless network 200. The control unit 113 of the single sensor device 100 may combine the parts received at different times through the update program to configure the updated deep-learning program.

Alternatively, the cloud server 500 may assign different portions to each of the sensor devices 100 of the group of the wireless network 200 (the group of sensor devices 100 connected to one wireless AP 300). Transmit through the wireless AP (300) (200).

Each sensor device 100 in a group of wireless network 200 has portions that send and receive each other to another sensor device 100 in a group of wireless network 200 and each of the sensor devices 100 (control 113 of) Can combine the parts through the update program to form an updated deep-learning program.

Subsequently, the control unit 113 of the sensor device 100 replaces the existing deep-learning program of the storage unit 107 with the updated deep-learning program through the update program (S70) and replaces the updated deep-learning program. The external event is monitored again, and the resulting data is generated from the image data or the sound data and transmitted to the cloud server 500 (see S10 to S40).

The controller 113 of the sensor device 100 may replace the existing deep learning program with a newly updated deep learning program in various ways. For example, the control unit 113 of the sensor device 100 allocates the nonvolatile memory area of the existing deep-learning program to the free memory area before replacing the non-volatile memory area of the newly updated deep-learning program. Set the start address and its size in the control program so that the updated deep-learning program can be executed later.

As such, the control unit 113 of the sensor device 100 (parts patched through the wireless network 200 at different times according to the promised format with the cloud server 500 or the wireless AP 300 (patch) Files are configured to complete one deep-learning program and replace the existing deep-learning program after completion. Substitution of the deep learning program may be performed while the deep learning program is running, or the deep learning program may be performed in a driving termination state.

The present invention described above is capable of various substitutions, modifications, and changes without departing from the technical spirit of the present invention for those skilled in the art to which the present invention pertains. It is not limited by the drawings.

100: sensor device
101: first sensor unit
103: second sensor unit
105: communication unit
107: storage unit
109: output unit
111: power supply
113: control unit
200: wireless network
300: wireless AP
400: broadband network
500: cloud server
600: external server

Claims (10)

A sensor device connected to a wireless IoT network to monitor the external environment,
A first sensor unit including a camera sensor;
A second sensor unit including one or more environmental sensors for sensing an environment external to the sensor device;
Communication unit for transmitting and receiving data through the IoT network;
A sensing program for recognizing an event designated by the second sensor unit, a deep-learning program for processing image data from the first sensor unit to recognize an object, and generating result data based on recognition of the object, the deep- A storage unit for storing an update program for updating a running program and a control program for controlling at least a driving and an end of the detection program, the deep-learning program, and the update program;
A controller configured to process image data from the first sensor unit through the deep-learning program to generate result data and to transmit the generated result data to a cloud server through the communication unit; And
A lithium ion battery or a battery is provided to supply power corresponding to a voltage level specified in the first sensor unit, the second sensor unit, the communication unit, the storage unit, and the control unit, and supply or cut power to the first sensor unit. It includes; a power supply having a power switch to control the;
The control unit receives sensing data from the second sensor unit through a sensing program, recognizes an external event by comparing the received sensing data with threshold data, and controls the first sensor according to the recognition of the external event through a control program. A control signal for supplying power to the unit is generated and output to the power supply unit to control driving of the first sensor unit, and recognize an object mapped to an external event from image data from the first sensor unit through the deep-learning program. Generates result data based on object recognition, maps the result data to image data, stores it in the storage unit, and transmits the generated result data to the cloud server through the communication unit,
The image data and the mapped result data stored in the storage unit are used as learning data for updating the deep-learning program in the cloud server.
Sensor device. The method of claim 1,
The IoT network is a LoRa network or an NB-IoT network,
The object is a person, a face, a vehicle, a vehicle number or a designated insect,
Sensor device. delete delete delete The method of claim 2,
The control unit for executing the update program may be configured to store portions of the deep-learning program updated by additional learning by a cloud server using image data and corresponding result data collected and stored by a plurality of sensor devices monitoring the same environment. Receive at different times from different sensor devices via a LoRa network or an NB-IoT network and combine the parts to replace the deep-learning program of the storage,
The controller allocates the memory area of the deep-learning program before updating to the free memory area, sets the address and the size of the storage memory area of the deep-learning program to be updated in the control program, and then updates the updated deep-learning program of the set memory area. To generate the resulting data,
Sensor device. The method of claim 1,
The storage unit includes a removable memory card,
The controller stores the image data and the mapped result data in the memory card.
Sensor device. The apparatus of claim 1, further comprising: a first sensor device; And a second sensor device. The method of claim 8,
And a cloud server that receives result data from the first sensor device and the second sensor device wirelessly via an IoT network and performs corresponding processing on each sensor device.
Environmental monitoring system. The method of claim 9,
The cloud server uses the offline image data and the result data collected by the first sensor device and the second sensor device as additional learning data to update the deep-learning program performed by the first sensor device and the second sensor device. Transmit each of the partial files of the updated deep-learning program to the first sensor device and the second sensor device,
The first sensor device configures the updated deep-learning program using the partial file received from the cloud server and the partial file received from the second sensor device through the IoT network and is stored in a storage unit. To replace a deep learning program with an updated deep learning program,
Environmental monitoring system.

Images