JP4556099B2 - Monitoring system, control device and method, and program - Google Patents

Description

The present invention is a monitoring system, control apparatus and method, and a program, more efficient monitoring system adapted monitoring process can be performed, the control apparatus and method, and a program.

  In recent years, with the improvement of information processing technology, information processing systems using various devices have been applied to various fields. For example, there is a monitoring system that uses an imaging device, a sound collection device, an information processing device, and the like to monitor an object that is present at a position where the user cannot visually recognize it (see, for example, Patent Document 1).

  The imaging device and the sound collection device are installed in, for example, another space different from the space where the user is located, and information acquired in the space is used as an information processing device installed in the space where the user is located using a communication device or the like. Supply. The information processing device that has acquired the information supplied by the imaging device and the sound collecting device creates output information by performing signal processing on the information, and outputs the output information from a display, a speaker, or the like. To do. The user grasps the situation of the space that cannot be directly visually recognized with the naked eye by referring to the output information.

  For example, as such a monitoring system, there is an infant monitoring system used by an infant to be monitored and a guardian of the infant to grasp the state of the infant. An infant monitoring system acquires an image or sound (crying) of an infant using a camera or a microphone in a space where the infant is located, and the image or sound is obtained via a display or a speaker in the space where the guardian is located. Output. By using such an infant monitoring system, the guardian can easily grasp the state of the infant located in another space with reference to the output image and sound.

JP-A-9-97392 (page 2-6, FIGS. 1 to 4)

  However, in the conventional monitoring system as described above, the processing method for detecting the state of the infant is fixed, and can be detected only by a predetermined method. Therefore, in the conventional monitoring system, the individuality of the infant to be monitored is not taken into consideration, and the state is detected and notified to any infant in the same manner, so that the infant's condition can be accurately monitored. There was a problem that it was not possible.

  Further, the request of the parent who is the user cannot be reflected in the process, and the optimum notification process cannot be performed for the infant cry desired by the user, so the number of useless notification processes increases. There was also a problem.

  Further, due to these, when the monitoring device including a camera, a microphone, and the like is driven by a battery, the operable time is shortened and becomes impractical. Therefore, the subject that the installation place of the monitoring apparatus will be limited to the place which can ensure a power supply occurred.

  The present invention has been made in view of such a situation, and makes it possible to perform monitoring processing more efficiently.

The monitoring system of the present invention is a monitoring system including a monitoring device that monitors a monitoring target and a main control device that controls the monitoring device , wherein the monitoring device acquires the image and sound of the monitoring target, and Generating notification information for notifying the state of the monitoring target, including images and sound, and supplying the generated notification information to the control device that presents the notification information to a user; The notification information supplied from the monitoring device is acquired, the sound of the monitoring target included in the acquired notification information is analyzed, the content of the notification information is determined, and the monitoring target is included in the notification information If it is determined that the image of the monitoring target is not included, the past notification information corresponding to the analysis result of the monitoring target sound included in the notification information that does not include the monitoring target image is included in the monitoring pair. Of searching the analysis results of the sound from the recording unit to record in association with the notification information, wherein the notification information of the past retrieved it is presented to the user.

The monitoring device detects an environmental change due to the monitoring target and supplies a detection result of the environmental change due to the monitoring target to the control device, and the control device acquires the detection result supplied from the monitoring device. If it is determined that the image and sound to be monitored are not included in the notification information, the past notification information corresponding to the acquired detection result is further associated with the detection result. It is possible to search from the recording unit for recording and present the searched past notification information to the user .

The control device receives designation of a sensor used by the user for detection of an environmental change by the monitoring target by the user, and supplies the received designation of the sensor as control information to the monitoring device, The monitoring device can acquire the control information supplied from the control device, select the sensor specified by the control information, and use it for detecting an environmental change due to the monitoring target .

The monitoring device and the control device can exchange the notification information via a dedicated line different from other information .

The control device accepts the creation or update of table information for associating the voice characteristics of the monitoring target and the state of the monitoring target by the user, or the specification of the table information, and is the newly created table information The table information is used to control the acquisition of the notification information based on the detection result, using the table information, the updated table information that is updated table information, or the designation of the table information as control information Supply to the monitoring device, the monitoring device acquires the control information supplied from the control device, and when the control information including the new table information is acquired, stores the new table information, the new table When control information including the update table information is acquired, the update is performed. Storing Buru information, the update table information set to use, if the control information including the designation of the table information is acquired can be set to use the table information specified by said control information .

The control device of the present invention is a control device that controls a monitoring device that monitors a monitoring target, and is information for notifying the status of the monitoring target supplied from the monitoring device , the image of the monitoring target, Notification information acquisition means for acquiring notification information including voice, presentation means for presenting the notification information to a user, and voice for analyzing the monitoring target voice included in the notification information acquired by the notification information acquisition means An analysis unit; a recording unit that records the result of analysis by the voice analysis unit in association with the notification information acquired by the notification information acquisition unit; a notification information determination unit that determines the content of the notification information; When the notification information determining unit determines that the monitoring target image is not included in the notification information, the voice analysis unit does not include the monitoring target image. Search means for searching past notification information corresponding to a result of analysis of the monitoring target voice included in the notification information from the notification information recorded by the recording means; When the notification information determination unit determines that the image to be monitored is not included in the notification information, the past notification information searched by the search unit is presented to the user. To do.

The apparatus further comprises detection result acquisition means for acquiring a detection result of an environmental change due to the monitoring target from the monitoring device, and the recording means further associates the detection result acquired by the detection result acquisition means with the notification information. And when the notification information determination means determines that the image and sound to be monitored are not included in the notification information, the search means adds the detection result to the detection result acquired by the detection result acquisition means. The corresponding past notification information is searched from the notification information recorded by the recording means, and the presentation means presents the past notification information searched by the search means to the user. Can do.

A footstep analysis unit that analyzes the footsteps of the user based on the notification information acquired by the notification information acquisition unit; and the user is positioned in the vicinity of the monitoring target based on an analysis result of the footsteps by the footstep analysis unit User position determination means for determining whether or not to perform the presentation, and when the user position determination means determines that the user is located in the vicinity of the monitoring target, the presentation means presents the notification information. Can be stopped .

Table information storage means for storing table information for associating the characteristics of the monitoring target voice with the monitoring target state, and the monitoring target by the voice analysis means using the table information acquired from the table information storage means State determination means for determining the state of the monitoring target from the result of the voice analysis, and the presentation means sends the notification information according to the state of the monitoring target determined by the state determination means. Whether to present to the user can be controlled .

When the creation of the table information by the user or the reception of the specification of the table information used by the state determination unit is received by the user, and the creation of the table information is received by the reception unit, The table information storage means stores the created table information, the state determination means is set to use the created table information, and when the update of the table information is accepted by the accepting means, The table information storage unit updates the stored table information, the state determination unit sets to use the updated table information, and when the specification of the table information is received by the reception unit, said state determining means, can be configured to use the specified the table information .

In order to control the acquisition of the notification information based on the detection result of the environmental change by the monitoring target, using the created table information, the updated table information, or the specification of the table information as control information Control information supply means for supplying to the monitoring device using table information can be further provided .

An accepting unit that accepts designation of a sensor used by the user for detection of an environmental change by the monitoring target by the user, and the designation of the sensor accepted by the accepting unit is supplied to the monitoring device as control information Control information supply means can be further provided .

The notification information acquisition means can acquire the notification information supplied from the monitoring device via a dedicated line different from other information .

The control method of the present invention includes a notification information acquisition step for acquiring notification information including a monitoring target image and sound, which is information supplied from a monitoring device for notification of the status of the monitoring target. By the processing of the voice analysis step for analyzing the voice to be monitored included in the notification information acquired by the processing of the information acquisition step, the notification information determination step of determining the content of the notification information, and the processing of the notification information determination step When it is determined that the monitoring target image is not included in the notification information, the monitoring target sound included in the notification information that does not include the monitoring target image by the processing of the voice analysis step. The past notification information corresponding to the result of the analysis is changed to the notification information acquired by the processing of the notification information acquisition step. A search step for searching from the notification information recorded by the recording unit that records the results of analysis by correlation, and a presentation for presenting to the user the past notification information searched by the processing of the search step And a step .

The program of the present invention is information for notifying the status of the monitoring target supplied from the monitoring device, the notification information acquiring step for acquiring the notification information including the image and sound of the monitoring target, and the notification information The voice analysis step of analyzing the monitoring target voice included in the notification information acquired by the processing of the acquisition step, the notification information determination step of determining the content of the notification information, and the processing of the notification information determination step When it is determined that the image to be monitored is not included in the notification information, analysis of the sound to be monitored included in the notification information that does not include the image to be monitored by the processing of the sound analysis step In the voice analysis step, the past notification information corresponding to the result of the notification information is acquired by the notification information acquisition step. A search step for searching from the notification information recorded by the recording unit that records the results of analysis by processing in association with each other, and a presentation for presenting to the user the past notification information searched by the processing of the search step And a step .

In the control device of the monitoring system of the present invention, the image and sound to be monitored are acquired, the notification information for notifying the state of the monitoring target including the image and sound is generated, and the generated notification information is notified. The information is supplied to the control device that presents the information to the user. In the control device, the notification information supplied from the monitoring device is acquired, the monitoring target audio included in the acquired notification information is analyzed, and the content of the notification information Past notification information corresponding to the analysis result of the monitoring target sound included in the notification information that does not include the monitoring target image when it is determined that the monitoring target image is not included in the notification information. However, it is searched from the recording unit that records the analysis result of the voice to be monitored in association with the notification information, and the past notification information searched is presented to the user .

In the control device and method, and the program of the present invention, information for notifying the state of the monitoring target supplied from the monitoring device is acquired and acquired including the monitoring target image and sound. The monitoring target audio included in the notification information is analyzed, the content of the notification information is determined, and if it is determined that the notification information does not include the monitoring target image, the notification that does not include the monitoring target image The past notification information corresponding to the result of analysis of the voice to be monitored included in the information is searched from the notification information recorded by the recording unit that records the analysis result in association with the acquired notification information, The searched past notification information is presented to the user .

  According to the present invention, a signal can be processed. In particular, the monitoring process can be performed more efficiently.

  Embodiments of the present invention will be described below with reference to the drawings.

  FIG. 1 is a block diagram showing a configuration example of a monitoring system to which the present invention is applied.

  In FIG. 1, a monitoring system 1 is an infant monitoring system in which, for example, an infant is a monitoring target and a guardian who monitors the state of the infant is a user. The monitoring apparatus 10 is installed in the vicinity of the infant and monitors the infant. The monitoring apparatus 10 is controlled, and is installed in the vicinity of the user. The monitoring apparatus 10 is configured by the main control apparatus 20 that acquires information about the infant from the monitoring apparatus 10 and notifies the user.

  The monitoring device 10 includes an antenna 11 for low-speed communication that transmits and receives control information and the like, and an antenna 12 for high-speed communication that transmits and receives notification information that is information about infants to be monitored. The main controller 20 also has an antenna 21 for low speed communication and an antenna 22 for high speed communication. The monitoring device 10 and the main control device 20 have a low communication speed such as IEEE (Institute of Electronic and Electronics Engineers) 802.11b or Bluetooth (Bluetooth (registered trademark)) via the antenna 11 and the antenna 21. Low-speed wireless communication is performed using the wireless communication standard, and high-speed wireless communication is performed via the antenna 12 and the antenna 22 using a wireless communication standard having a high communication speed such as IEEE802.11a or IEEE802.11g.

  IEEE802.11x is a standard standardized by IEEE (American Institute of Electrical and Electronics Engineers) that promotes the standardization of wireless local area networks (LANs), such as the frequency band and modulation method used as a base for communication. This defines the physical layer and the medium access control such as data format and communication method. IEEE802.11b is an extended standard for a wireless LAN that uses a 2.4 GHz band and performs wireless communication in a DSSS (Direct Sequence Spectrum Spread) method (direct spread spectrum spread) at a maximum transfer rate of 11 Mbps. In addition, IEEE802.11a uses OFDM (Orthogonal Frequency Division Multiplex), which uses a 5 GHz band, transmits data using a plurality of narrowband carriers at a transfer rate of 54 Mbps at maximum, and performs wireless communication using a modulation method called OFDM (Orthogonal Frequency Division Multiplex). An extended standard for wireless LAN. Furthermore, it is an extended standard of wireless LAN that uses the 2.4 GHz band of IEEE802.11g and performs OFDM modulation wireless communication at a maximum transfer rate of 54 Mbps. Bluetooth is a wireless transmission method using a 2.4 GHz band and a maximum transmission rate of 1 Mbps.

  As will be described later, the monitoring device 10 supplies the output of a sensor, which will be described later, to the main control device 20 by low-speed wireless communication via the antenna 11 and the antenna 21, and also acquired via the built-in camera or microphone. Notification information about the infant is supplied to the main controller 20 by high-speed wireless communication via the antenna 12 and the antenna 22. The main controller 20 outputs the sensor output and the notification information about the infant acquired in this way from a display or the like. In addition, the main control device 20 supplies control information created based on, for example, a user's instruction to the monitoring device 10 by low-speed wireless communication via the antenna 21 and the antenna 11 and performs the operation of the monitoring device 10. Control.

  A user who is a guardian of an infant can grasp the state of the infant by referring to the information output from the main controller 20.

  FIG. 2 is a diagram illustrating a detailed configuration example of the monitoring apparatus 10 of FIG.

  The monitoring device 10 includes a control unit 31 that controls each unit of the monitoring device 10, a RAM (Random Access Memory) 32 that is a semiconductor memory that temporarily stores programs and data executed by the control unit 31, and is executed by the control unit 31. ROM (Read Only Memory) 33 which is a read only semiconductor memory in which programs and data to be recorded are recorded.

  The control unit 31 is configured by a CPU including a calculation unit, a control unit, and a holding unit, executes programs and data stored in the RAM 32 and the ROM 33, controls each unit of the monitoring device 10, and performs monitoring. A process of acquiring an image or sound of an infant and supplying it to the main control device 20 or setting the monitoring device 10 is performed.

  The monitoring device 10 includes a plurality of sensors, a sensor unit 41 that acquires information about the infant's condition, a low-speed communication unit 42 that performs low-speed wireless communication with the main controller 20 via the antenna 11, an image of the infant, Notification information acquisition unit 43 that acquires information to be notified to the user (hereinafter referred to as notification information), such as voice, signal processing or the like is performed on the notification information supplied from notification information acquisition unit 43, and notification information for transmission is provided. Lookup information processing unit 44 to be generated, a high-speed communication unit 45 that performs high-speed wireless communication with the main control device 20 via the antenna 12, and a lookup that the control unit 31 refers to to control each unit according to the state of the infant and the like The operation of each unit of the monitoring device 10 is controlled by the LUT storage unit 46 that stores the table (LUT), the power supply unit 47 that supplies power to each unit of the monitoring device 10, and the control unit 31. And a work control unit 48.

  The sensor unit 41 includes three sensors that measure different information of the first sensor 51, the second sensor 52, and the third sensor, and further, the operations of the first sensor 51 to the third sensor 53 are respectively performed. The sensor driver 54 thru | or sensor driver 56 to control, and the selector 57 which selects the sensor output supplied to the control part 31 from the sensor output of the 1st sensor 51 thru | or the 3rd sensor 53 are provided.

  Although not shown, the first sensor 51 is composed of a simple microphone for acquiring a voice uttered by an infant and is controlled by the sensor driver 54 to output the acquired voice signal to the sensor driver 54 as an output signal. . Although not shown, the sensor driver 54 includes an amplifier circuit that amplifies the acquired output signal of the microphone and a measurement unit that measures and outputs a predetermined parameter from the amplified output signal. The output signal output by controlling one sensor 51 is amplified, the infant's breathing time and utterance time are measured, and the measured values are supplied to the selector 57 as sensor outputs.

  Although not shown in the drawings, the second sensor 52 is composed of an air pressure sensor that detects a change in space pressure due to a voice uttered by an infant and converts it into an electric signal, and is controlled by the sensor driver 55 to change the air pressure. The electrical signal obtained by converting the signal is output to the sensor driver 55 as an output signal. Although not shown, the sensor driver 55 includes an amplifier circuit that amplifies the output signal of the second sensor 52, and a measurement unit that measures and outputs a predetermined parameter from the amplified output signal. The output signal outputted by controlling the second sensor 52 is amplified, the infant's breathing time and utterance time are measured, and the measured value is supplied to the selector 57 as a sensor output.

  Although not shown, the third sensor 53 is an infrared (temperature) sensor that detects a change in the body temperature of the infant and converts the change into an electrical signal, and is controlled by the sensor driver 56 to convert the change in the body temperature. The electrical signal obtained in this way is output to the sensor driver 56 as an output signal. Although not shown, the sensor driver 56 includes an amplifier circuit that amplifies the output signal of the third sensor 53, and a measurement unit that measures and outputs a predetermined parameter from the amplified output signal. An output signal output by controlling the third sensor 53 is amplified, the infant's breathing time and utterance time are measured, and the measured value is supplied to the selector 57 as a sensor output.

  The selector 57 is controlled by the control unit 31 and selects any one of the sensor outputs of the first sensor 51 to the third sensor 53 supplied from the sensor driver 54 to the sensor driver 56, and sends it to the control unit 31. Supply.

  As described above, the low-speed communication unit 42 communicates with the main controller 20 of FIG. 1 via the antenna 11 by a low-speed wireless communication method such as IEEE802.11b or Bluetooth, and is supplied from the main controller 20. When control information including sensor designation information, information about a table, and the like, which will be described later, is acquired, the control information is supplied to the control unit 31. Further, the low speed communication unit 42 supplies the sensor output supplied from the sensor unit 41 via the control unit 31 to the main controller 20.

  The notification information acquisition unit 43 includes an imaging unit 61 that images an infant and outputs the obtained image data, and a sound collection unit 62 that collects an infant's cry and outputs the obtained audio data. Notification information including image data and sound data of the infant is output (output to the user) from the display or speaker of the control device 20.

  FIG. 3 is a diagram illustrating a detailed configuration example inside the notification information acquisition unit 43. In FIG. 3, the imaging unit 61 is configured by a camera 83 or the like that is installed in a direction in which an infant 84 that is a subject can be imaged, and images the infant 84 that is a monitoring target. ) Or CMOS (Complementary Metal Oxide Semiconductor) or the like, a captured image is converted into an electrical signal by a photoelectric element (not shown) and supplied to the notification information processing unit 44 in FIG. 2 as image data. The sound collection unit 62 includes two microphones 81 and microphones 82 installed at predetermined intervals at positions on both sides of the camera 83 in the lateral direction, and collects sounds mainly uttered by the infant 84. And supplied to the notification information processing unit 44 as audio data. Note that the microphone 81 and the microphone 82 collect unnecessary sound such as sound uttered by another person 85 located near the infant 84 in addition to the sound uttered by the infant 84. Therefore, as will be described later, the sound collection unit 82 calculates and uses the delay amount between the audio signal obtained by the microphone 81 and the audio signal obtained by the microphone 82, so that the infant 84 to be monitored is used. 2 and the position of another person 85 that is not monitored, located near the infant 84, and two unnecessary microphones are used so that unnecessary noise components can be removed. To collect sound.

  Returning to FIG. 2, the notification information processing unit 44 that has acquired image data from the imaging unit 61 of the notification information acquisition unit 43 or acquired audio data from the sound collection unit 62 performs image processing on the acquired image data. In addition, by performing signal processing on the acquired audio data, the notification information for transmission is generated and supplied to the high-speed communication unit 45.

  As described above, the high-speed communication unit 45 communicates with the main controller 20 of FIG. 1 via the antenna 12 by a high-speed wireless communication method such as IEEE802.11a or IEEE802.11g, and the notification information processing unit 44. The supplied notification information is supplied to the main controller 20.

  The LUT storage unit 46 is configured by a semiconductor memory or the like, and stores a feature value table which is a LUT (Look Up Table) for controlling the operation of the monitoring device 10 based on the feature value obtained from the sensor output.

  The LUT storage unit 46 includes a non-volatile memory such as ROM or EEPROM (Electronically Erasable and Programmable Read Only Memory), for example, and includes a basic LUT storage unit 71 that stores LUTs recorded at the time of factory shipment, and, for example, An update LUT storage unit 72 that includes a flash memory, a RAM, and the like and stores a new feature amount table supplied from the main controller 20 of FIG. The table (LUT) is searched from the feature value table (LUT) stored in the basic LUT storage unit 71 or the updated LUT storage unit 72 and supplied to the control unit 31. Further, the update table supplied from the control unit 31 is stored in the update LUT storage unit 72.

  The power supply unit 47 includes, for example, a power adapter, a dry battery, or a storage battery, and is controlled by the operation control unit 48 to supply power to each unit of the monitoring device 10. In FIG. 2, the main power supply destination by the power supply unit 47 is indicated by an arrow, but the power supply unit 47 also supplies power to other units.

  The operation control unit 48 is controlled by the control unit 31 being supplied with the state value of the infant 84 calculated by referring to the table 91, and based on the supplied state value, the sensor unit 41, It controls the start and end of operations of the low-speed communication unit 42, the notification information acquisition unit 43, the notification information processing unit 44, the high-speed communication unit 45, and the like. The operation control unit 48 also controls the power supply unit 47 to detect the sensor unit 41, the low-speed communication unit 42, the imaging unit 61 and the sound collection unit 62 of the notification information acquisition unit 43, the notification information acquisition unit 44, and the high-speed The start and stop of the supply of power to the communication unit 45 are controlled. In FIG. 2, the main control destination of the operation by the operation control unit 48 is indicated by an arrow, but the operation control unit 48 may also control the operation of each other unit.

  Next, an internal configuration example of the main controller 20 in FIG. 1 will be described. FIG. 4 is a block diagram illustrating an internal configuration example of the main controller 20 of FIG.

  In FIG. 4, the main control device 20 is executed in the control unit 101 that controls the operation of each unit of the main control device 20, the RAM 102 that temporarily holds programs and data executed in the control unit 101, and the control unit 101. It has a ROM 103 in which programs and data are stored.

  The control unit 101 includes a CPU including a calculation unit, a control unit, and a holding unit, executes programs and data stored in the RAM 102 and the ROM 103, controls each unit of the main control device 20, and monitors the monitoring device The information regarding the infant 84 supplied from 10 is acquired, the acquired information is analyzed and output, or the operation of the monitoring device 10 is controlled based on user input or the like.

  In addition, the main control device 20 includes a low-speed communication unit 111 that performs low-speed wireless communication with the monitoring device 10 via the antenna 21 and a high-speed communication unit 112 that performs high-speed wireless communication with the monitoring device 10 via the antenna 22. The filter bank 113 that performs processing for removing noise components from the voice data of the notification information acquired by the high-speed communication unit 112, the notification information storage unit 114 that stores the notification information in a database, and the notification information storage unit 114 The notification information search unit 115 for searching for desired notification information from the accumulated past notification information, and the past notification information searched by the notification information search unit 115 and the current notification information supplied from the monitoring device 10 are combined. Any one of the combination processing unit 116, the notification information combined by the combination processing unit 116, and the current notification information supplied from the monitoring device 10 A footstep analysis unit 121 that performs speech analysis on the speech data included in the notification information from which noise components have been removed by the selector 117 that selects the notification information to be output and the filter bank 113 and identifies the footstep of the user, a filter The voice analysis is performed on the voice data included in the notification information from which the noise component has been removed by the bank 113, and extracted by the voice analysis unit 122 and the voice analysis unit 122 that extract the feature amount of the cry of the infant to be monitored. A qualitative value determination unit 123 that identifies the type of cry desired by the user based on the feature amount of the infant's cry and user input, a LUT storage unit 131 that stores the LUT that controls the operation of the monitoring device 10, and accepts user input Controlled by the input unit 141 and the control unit 101, a simple display unit 142 for displaying a message or the like indicating the state of the infant, And a notification data output unit 143 outputs the notification information is selected as information to be output by Kuta 117.

  The low-speed communication unit 111 performs wireless communication with the low-speed communication unit 42 of the monitoring device 10 illustrated in FIG. 2 by using a low-speed wireless communication method such as IEEE802.11b or Bluetooth, and acquires the sensor output supplied from the monitoring device 10 Then, the control information is supplied to the control unit 101 or the control information supplied from the control unit 101 is supplied to the monitoring device 10. The control information supplied to the monitoring device 10 includes, for example, table number designation information, an update table, or sensor designation information.

  The high-speed communication unit 112 performs wireless communication with the high-speed communication unit 45 of the monitoring apparatus 10 illustrated in FIG. 2 using a high-speed wireless communication method such as IEEE802.11a or IEEE802.11g, and notification information supplied from the monitoring apparatus 10 To get.

  The filter bank 113 includes various units as shown in FIG. 5 and performs a process of removing noise components (voices other than the cry uttered by the infant) of the acquired notification information. In FIG. 5, the filter bank 113 converts the audio data supplied from the high-speed communication unit 112 into notification information from which high-frequency components are removed by the low-pass filter 161 and the low-pass filter 161 that remove high-frequency components of a predetermined frequency or higher. A noise component removal processing unit 162 that performs processing for removing noise components is provided.

  The noise component removal processing unit 162 includes a component of the sound data collected by the microphone 81 and a component of the sound data collected by the microphone 82 included in the sound data from which the high frequency component has been removed by the low-pass filter 161. A delay amount determination unit 171 that determines a delay amount between them, and corrects the audio data using the delay amount. In the audio data corrected by the delay amount determination unit 171, the component of the audio data collected by the microphone 81 and In the audio data corrected by the delay correction addition processing unit 172 and the delay amount determining unit 171 for adding the audio data components collected by the microphone 82, the audio data components collected by the microphone 81 and the microphone The delay correction subtraction processing unit 173 that subtracts the sound data components collected by 82 and the delay correction subtraction processing unit 173 reduce A moving average processing unit 174 that calculates a moving average from the audio data that has been added, an addition processing unit 175 that adds the audio data added by the delay correction addition processing unit 172 and the audio data subtracted by the delay correction subtraction processing unit 173, and addition Correlation coefficient processing unit 176 that calculates a correlation coefficient based on the audio data added in the processing unit 175, a corresponding region that analyzes the content of the audio data and determines whether or not the noise component is removed The determination unit 177 and the corresponding region determination unit 177 include an attenuation processing unit 178 that performs noise component attenuation processing in a region determined to be a portion from which the noise component is removed.

  That is, the filter bank 113 not only limits the frequency band of the audio data included in the notification information using a filter, but also performs a process of removing noise components other than the cry of the infant 84.

  Returning to FIG. 4, the notification information storage unit 114 is configured by, for example, a magnetic recording medium such as a hard disk, an optical disk, a magneto-optical disk, or a semiconductor memory, and is controlled by the control unit 101 and supplied from the high-speed communication unit 112. The sensor output supplied via the control unit 101, the analysis result of the voice analysis unit 122, and the determination result of the qualitative value determination unit 123 are stored in association with the notification information.

  The notification information search unit 115 searches the information stored in the notification information storage unit 114 for a situation similar to the situation based on the information about the current situation of the monitoring target supplied by the control unit 101, The past notification information corresponding to the closest situation or the closest situation is acquired, and the acquired past notification information is supplied to the synthesis processing unit 116.

  The combination processing unit 116 is controlled by the control unit 101 to combine the past notification information supplied from the notification information search unit 115 with the current notification information supplied from the high-speed communication unit 112, and the combined notification information Is supplied to the selector 117. The selector 117 selects either notification information supplied from the synthesis processing unit 116 or notification information supplied from the high-speed communication unit 112 based on an instruction from the control unit 101, and outputs notification information. Part 143.

  The input unit 141 includes an input interface such as a keyboard, a mouse, a touch pen, a camera, or a microphone, receives an input from a user, and supplies the user input to the control unit 101.

  The simple display unit 142 is configured by a small display such as an LCD (Liquid Crystal Display), etc., and is controlled by the control unit 101 to supply information (for example, characters such as a message indicating an infant's status). Information).

  The notification information output unit 143 includes a monitor 151, a speaker 152, and the like, and is controlled by the control unit 101 to output the notification information supplied from the selector 117 using the monitor 151 and the speaker 152.

  Next, the operation of the monitoring system 1 in FIG. 1 will be described.

  The control unit 31 of the monitoring device 10 controls the operation of the sensor unit 41 based on the sensor output supplied from the sensor unit 41, the control information supplied from the main control device 20, and the like, as will be described later.

  In the initial state, the control unit 31 controls the sensor driver 54 and the like of the sensor unit 41, obtains an audio signal using the first sensor 51, measures a breathing time and an utterance time based on the audio signal, and selects a selector. The measurement result is supplied to the control unit 31 through 57 as a sensor output. The selector 57 is controlled by the control unit 31 to switch the input so that the measurement values (breathing time and utterance time) supplied from the sensor driver 54 are supplied to the control unit 31.

  At that time, the control unit 31 controls the power supply unit 47 via the operation control unit 48 to supply power to the first sensor 51, the sensor driver 54, and the selector 57 of the sensor unit 41, and the second other than that. The supply of power to the sensor 52, the third sensor 53, the sensor driver 55, and the sensor driver 56 is stopped. By stopping the supply of power to unnecessary circuits in this way, power consumption can be suppressed. As a result, the monitoring device 10 can operate more efficiently, such as being able to operate for a longer time, even when driven by a battery.

  It should be noted that the first sensor 51 normally has the least power consumption compared to the second sensor 52 and the third sensor, and can sense most accurately when the sound collection environment is good. However, for example, in the case where the environmental sound (unnecessary noise component other than the crying voice of the infant 84) is large, the sensor driver 54 outputs a voice signal that includes a large amount of the noise component output from the first sensor 51. There is a risk that an inaccurate measurement result may be output as a sensor output because the waveform of the cry cannot be measured with sufficient accuracy.

  By the way, it is widely known that the breathing time of a general infant's cry is about 200 msec to 650 msec, and the utterance time is about 200 msec to 1000 msec. Accordingly, when the sensor driver 54 frequently measures an audio pattern outside such a range, the control unit 31 includes an environmental sound in an audio signal output from the first sensor 51 that exceeds an allowable amount. In this case, the sensor to be used is switched to the second sensor 52 to measure the breathing time and the utterance time.

  At that time, the control unit 31 controls the power supply unit 47 via the operation control unit 48 to supply power to the second sensor 52, the sensor driver 55, and the selector 57 of the sensor unit 41, and the other first The supply of power to the sensor 51, the third sensor 53, the sensor driver 54, and the sensor driver 56 is stopped. By stopping the supply of power to unnecessary circuits in this way, power consumption can be suppressed. As a result, the monitoring device 10 can operate more efficiently, such as being able to operate for a longer time, even when driven by a battery.

  The second sensor 52 is an air pressure sensor and requires a sensor amplifier or a charge amplifier having a large gain. Therefore, the second sensor 52 consumes power compared to the first sensor 51, but detects a precise air flow. be able to. The change in the air pressure detected by the second sensor 52 is supplied as an electrical signal to the sensor driver 55 as in the case where the first sensor 51 is used. The sensor driver 55 measures the breathing time and the utterance time based on the acquired electrical signal, and supplies the measurement result to the control unit 31 via the selector 57 as a sensor output. The selector 57 is controlled by the control unit 31 to switch the input so that the measurement values (breathing time and utterance time) supplied from the sensor driver 55 are supplied to the control unit 31.

  However, for example, in the space where the infant 84 is located, the air pressure may change due to other factors such as wind. In such a case, the sensor driver 55 determines that the electrical signal output from the second sensor 52 is There is a risk that an inaccurate measurement result may be output as a sensor output because the waveform of the cry cannot be measured with sufficient accuracy.

  Therefore, when a pressure value that exceeds a predetermined threshold value is measured in the electrical signal supplied to the sensor driver 55, the control unit 31 converts the electrical signal output from the second sensor 52 into the electrical signal. It is determined that the pressure change due to factors other than the crying voice of the infant 84 exceeds the allowable amount. In that case, the sensor to be used is switched to the third sensor 53 to measure the breathing time and the utterance time.

  At that time, the control unit 31 controls the power supply unit 47 via the operation control unit 48 to supply power to the third sensor 53, the sensor driver 56, and the selector 58 of the sensor unit 41, and the other first The supply of power to the sensor 51, the second sensor 52, the sensor driver 54, and the sensor driver 55 is stopped. By stopping the supply of power to unnecessary circuits in this way, power consumption can be suppressed. As a result, the monitoring device 10 can operate more efficiently, such as being able to operate for a longer time, even when driven by a battery.

  The third sensor 53 is a sensor that detects a temperature change due to respiration, and is not affected by environmental sound or wind, but requires more power than the first sensor 51 and the second sensor 52.

  In addition, when the control unit 31 acquires the control information supplied from the main control device 20 via the low-speed communication unit 42, the control unit 31 determines a sensor to be used based on the sensor designation information included in the control information.

  At that time, the control unit 31 controls the power supply unit 47 via the operation control unit 48 and causes the sensor unit 41 to supply power only to the part related to the sensor to be used, and to the other units of the sensor unit 41. Stop supplying power. By stopping the supply of power to unnecessary circuits in this way, power consumption can be suppressed. As a result, the monitoring device 10 can operate more efficiently, such as being able to operate for a longer time, even when driven by a battery.

  When power is supplied to the monitoring device 10 and power is supplied to the control unit 31, the control unit 31 performs sensor switching processing to switch the sensor to be used as described above. This sensor switching process will be described with reference to the flowcharts of FIGS.

  First, in step S1, the control unit 31 determines whether or not the first sensor 51 is operating via the sensor driver 54. If it is determined that the first sensor 51 is not operating, the process proceeds to step S2. In step S <b> 2, the control unit 31 controls the power supply unit 47 via the operation control unit 48 to start supplying power to the first sensor 51 and to start supplying power to the sensor driver 54 and the selector 57. Further, the supply of electric power to each part (the second sensor 52, the third sensor 53, the sensor driver 55, and the sensor driver 56) that is not operated by the sensor unit 41 is stopped. And the control part 31 controls the sensor driver 54 in step S3, and starts the measurement process by the 1st sensor 51. FIG. The control part 31 which started the measurement process by the 1st sensor 51 advances a process to step S4. If it is determined in step S1 that the first sensor 51 is operating, the control unit 31 omits steps S2 and S3 and proceeds to step S4.

  In step S4, the control unit 31 measures the utterance time and breathing time measured by the sensor driver 54 (the breathing time is measured between about 200 msec and 650 msec, and the utterance time is measured between about 200 msec and 1000 msec. To be satisfied), and based on the determination result, it is determined whether or not the output of the first sensor 51 can be analyzed. When the utterance time and the breathing time measured by the sensor driver 54 are frequently measured outside the above-described range, the above-described conditions are not satisfied, and it is determined that the output of the first sensor 51 cannot be analyzed. The unit 31 determines that the environmental signal is included in the audio signal output from the first sensor 51 in an allowable amount or more, and switches the sensor to be used to the second sensor 52 to measure the breathing time and the utterance time. That is, in this case, the control unit 31 proceeds to step S5, controls the sensor driver 54 to end the measurement process by the first sensor 51, and in step S6, the power supply unit 47 via the operation control unit 48. And the supply of power to the first sensor 51 and the sensor driver 54 is stopped. In step S7, the power supply unit 47 is controlled via the operation control unit 48, and the power supply to the second sensor 52 and the sensor driver 55 is controlled. In step S8, the sensor driver 55 is controlled to start measurement processing by the second sensor 52, and the process proceeds to step S9.

  In step S9, the control unit 31 determines whether or not the sensor driver 55 measures a pressure value that exceeds a predetermined threshold value, and the output of the second sensor 52 is based on the determination result. Whether or not can be analyzed is determined. When the sensor driver 55 measures a pressure value exceeding a predetermined threshold value and determines that the output of the second sensor 52 cannot be analyzed, the control unit 31 outputs the pressure value from the second sensor 52. It is determined that the pressure change due to factors other than the crying voice of the infant 84 is included in the electrical signal that exceeds the allowable amount, and the sensor to be used is switched to the third sensor 53 to measure the breathing time and the utterance time. That is, in this case, the control unit 31 advances the process to step S10, controls the sensor driver 55 to end the measurement process by the second sensor 52, and in step S11, the power supply unit 47 via the operation control unit 48. And the power supply to the second sensor 52 and the sensor driver 55 is stopped. In step S12, the power supply unit 47 is controlled via the operation control unit 48, and the power supply to the third sensor 53 and the sensor driver 56 is controlled. In step S13, the sensor driver 56 is controlled to start measurement processing by the third sensor 53, and the process proceeds to step S21 in FIG.

  In step S <b> 201 of FIG. 7, the control unit 31 determines whether or not the sensor driver 56 can analyze the output of the third sensor 53. When the sensor driver 56 measures a temperature change exceeding a predetermined threshold value and determines that the output of the third sensor 53 cannot be analyzed, the control unit 31 outputs the third sensor 53. It is determined that the temperature change due to factors other than the infant 84 generating a cry is included in the electrical signal exceeding the allowable amount, and the measurement of the breathing time and the utterance time by the sensor is stopped. That is, in this case, the control unit 31 proceeds to step S22 to control the sensor driver 56 to end the measurement process by the third sensor 53. In step S23, the power supply unit 47 via the operation control unit 48 is performed. And the supply of power to the third sensor 53, sensor driver 54, and selector 57 is stopped (that is, the supply of power to the sensor unit 41 is stopped), and error processing is performed in step S24. After proceeding to S25 and performing an end process, the sensor switching process is ended.

  In addition, when it determines with the output of the 1st sensor 51 being analyzable in step S4 of FIG. 6, when it determines with the output of the 2nd sensor 52 being analyzable in step S9 of FIG. 6, or in step S21 of FIG. When it is determined that the output of the third sensor 53 can be analyzed, in each case, the control unit 31 proceeds with the process to step S25 and performs an end process in order to perform a measurement process using each sensor. Then, the sensor switching process ends.

  As described above, the control unit 31 performs the sensor switching process, causes the infant 84 to measure the breathing time and the utterance time using an optimum sensor, and acquires the sensor output including them. This sensor switching process is repeated at predetermined intervals by the control unit 31 while the monitoring device 10 is monitoring the infant 84, and an optimum sensor is selected in the state at that time.

  In this way, the control unit 31 switches the sensor of the sensor unit 41 according to the environment, and operates the sensor unit 41 so as to react to a desired cry with appropriate and minimum necessary power. As a result, the monitoring device 10 can operate more efficiently, such as being able to operate for a longer time, even when driven by a battery.

  Further, the control unit 31 stores the basic LUT storage unit 71 of the LUT storage unit 46 based on the table number designation information included in the control information supplied from the main control device 20 acquired through the low-speed communication unit 42. The feature quantity table to be used is determined from the feature quantity tables being used. In addition, when the control information acquired from the low-speed communication unit 42 and supplied from the main control device 20 includes an update table that is a new feature amount table, the control unit 31 converts the update table to the LUT. The data is supplied to the storage unit 46, stored in the update LUT storage unit 72, and held in the RAM 32. As will be described later, the control unit 31 determines the state value of the infant 84 by referring to the update table, and supplies the state value to the operation control unit 48.

  The control unit 31 acquires the feature amount table from the LUT storage unit 46 based on the sensor output supplied from the sensor unit 41 and the control information supplied from the main control device 20 as described above, and from the feature amount table. Each unit is appropriately controlled in accordance with the calculated state value of the infant 84 to cause the main control device 20 to supply notification information such as image data and audio data.

  Specifically, when power is supplied to the monitoring device 10 and power is supplied to the control unit 31, the control unit 31 controls the sensor unit 41 to supply sensor output and execute monitoring control processing. Then, each unit is controlled based on the acquired sensor output and the feature amount table acquired from the LUT storage unit 46. The monitoring control process will be described with reference to the flowchart of FIG. Further, description will be made with reference to FIG. 9 as necessary.

  First, in step S31, the control unit 31 determines whether or not to end the monitoring control process. When it determines with not complete | finishing, the control part 31 advances a process to step S32, controls the sensor part 41, acquires a sensor output, analyzes the acquired sensor output in step S33, and respond | corresponds to a feature-value table. Parameters such as speaking time and breathing time are generated. The control unit 31 that has generated the parameter refers to the feature amount table acquired from the LUT storage unit 46 in step S34.

  FIG. 9 is a diagram illustrating an example of the LUT stored in the LUT storage unit 46. The feature value table 181 shown in FIG. 9 is a table in which two feature values of the cry of the infant 84 and the state of the infant 84 are associated with each other in a matrix. The column of the feature amount table 181 indicates the length of the breathing time (silent time) of the infant 84 in FIG. 3 as the monitoring target, and the row indicates the length of the utterance time of the infant 84 as the monitoring target. In the feature amount table 181, the states of the infants 84 are grouped into four stages of values “0” to “3” according to the length of the breathing time and the utterance time, and the length of the breathing time and the utterance time. This value (hereinafter referred to as state value) is calculated based on the above. As will be described later, the operation control unit 48 determines the content of information to be supplied to the main controller 20 based on this state value.

  That is, in step S34, the control unit 31 calculates the state value of the infant 84 using the feature amount table 181 from parameters such as a speech time and a breathing time obtained from the sensor output output from the sensor unit 41. .

  The control unit 31 that has calculated the state value of the infant 84 supplies the state value to the operation control unit 48 in step S35, thereby controlling the operation of each unit based on the state value.

  The control part 31 which complete | finished the process of step S35 returns a process to step S31, and repeats the process after it. That is, the control unit 31 repeatedly executes the processes of steps S31 to S35, and controls each unit based on the sensor output supplied from the sensor unit 41. In step S31, for example, based on a user input or the like input via an input unit (not shown), the monitoring control process is performed to turn off the power of the monitoring device 10 or shift to the suspended state. If it is determined that the process ends, the control unit 31 proceeds to step S36, performs the end process, and then ends the monitoring control process.

  Incidentally, the feature value table 181 in FIG. 9 is a table for determining the state value of the infant 84 based on the breathing time (silence time) and the utterance time. In general, the meaning of the infant cry is the breathing time. It is known that it can be classified well by utterance time or its ratio. Therefore, by setting the state value in the feature value table 181 based on the meaning of the infant's cry, the monitoring apparatus 10 determines only the desired cry based on the feature value table 181; It is also possible to transmit image data, audio data, and the like to the main controller 20.

  The feature amount of the crying voice of the infant 84 constituting the feature amount table 181 may be a feature amount other than those described above, and the number of feature amounts may be any number.

  When the control unit 31 acquires the feature amount table 181 to be used from the LUT storage unit 46, the control unit 31 stores the feature amount table 181 in the RAM 32 or a holding unit built in the control unit 31, and is supplied from the sensor unit 41 as described above. The state value of the infant 84 is determined using the feature amount table 181 from the sensor output (the length of the breathing time and the utterance time of the infant 84), and the state value is supplied to the motion control unit 48. The operation control unit 48 controls operations of the power supply unit 47, the notification information acquisition unit 43, the notification information processing unit 44, the high-speed communication unit 45, the sensor unit 41, and the low-speed communication unit 42 based on the state value.

  For example, when the state value is “3”, the operation control unit 48 controls the power supply unit 47 to perform the imaging unit 61 and the sound collection unit 62 of the notification information acquisition unit 43, the notification information processing unit 44, and the high-speed communication. The unit 45 is activated by supplying power, and is controlled to transmit image data and audio data to the main controller 20 as notification information.

  Accordingly, the imaging unit 61 of the notification information acquisition unit 43 is controlled by the operation control unit 48 to image the infant 84 to be monitored, acquire an image of the infant 84 as image data, and supply the image data to the notification information processing unit 44. To do. The sound collection unit 62 of the notification information acquisition unit 43 is controlled by the operation control unit 48 to acquire, as voice data, a cry uttered by the infant 84 to be monitored (sound in the space where the infant 84 is located) as voice data. The information is supplied to the information processing unit 44.

  When the notification information processing unit 44 is controlled by the operation control unit 48 and acquires the supplied image data and audio data, the notification information processing unit 44 performs image processing and signal processing on the received image data and signal data, generates notification information for transmission including them, and performs high-speed processing. Supply to the communication unit 45. The high-speed communication unit 45 communicates with the main control device 20 via the antenna 12 and supplies the supplied notification information including image data and audio data to the main control device 20.

  When the acquired state value is “2”, the operation control unit 48 controls the power supply unit 47 to collect the sound collection unit 62, the notification information processing unit 44, and the high-speed communication unit 45 of the notification information acquisition unit 43. Is activated, and only the audio data is transmitted to the main controller 20. At this time, the operation control unit 48 ends the process being executed by the imaging unit 61 and controls the power supply unit 47 to stop the supply of power to the imaging unit 61. In this case, since the amount of information to be transmitted is small, the operation time of the high-speed communication unit 45 is shortened compared to the case where the above-described state value is “3”, so the monitoring device 10 suppresses power consumption. Can be made.

  By the way, when the acquired state value is “1” or “0”, the operation control unit 48 includes the imaging unit 61 and the sound collection unit 62 of the notification information acquisition unit 43, the notification information processing unit 44, and the high-speed communication unit 45. While stopping the operation, the power supply unit 47 is controlled to stop the supply of power to these units.

  When the state value is “1” to “3”, the operation control unit 48 further controls the power supply unit 47 to supply power to the sensor unit 41 and the low-speed communication unit 42 to start up the sensor. Supply output or receive control signals.

  In this way, the operation control unit 48 is controlled by the control unit 31 to operate only the minimum necessary circuit. Thereby, the monitoring apparatus 10 can implement | achieve reduction in power consumption, and can operate | move more efficiently, such as being able to operate | move for a long time, even when driving a battery as a power supply.

  The operation control unit 48 to which the state value is supplied from the control unit 31 starts executing the operation control process when the monitoring device 10 is powered on. When the state value is supplied from the control unit 31, the operation control unit 48 controls the operation of each unit based on the state value as described above. The operation control process will be described with reference to the flowcharts of FIGS.

  First, the operation control unit 48 determines whether or not the state value supplied from the control unit 31 is “0” in step S51 of FIG. 10 and determines that the state value is “0”. Then, the process proceeds to step S52 in order to control each part of the monitoring control apparatus 10 so as not to provide notification information or perform processing such as low-speed wireless communication with the main control apparatus 20.

  In step S <b> 52, the operation control unit 48 is a process for supplying notification information to the main control device 20 in each unit of the monitoring device 10, which will be described later, a first notification information provision process and a second notification information provision process. Alternatively, it is determined whether or not low-speed communication processing for performing low-speed wireless communication with the main controller 20 is being executed. When it is determined that the notification information providing process (first notification information providing process or second notification information providing process) or the low-speed communication process is being executed, the operation control unit 48 performs the notification information providing process and the In step S54, the power supply unit 47 is controlled to supply power to the imaging unit 61, the sound collection unit 62, the notification information processing unit 44, the high-speed communication unit 45, and the low-speed communication unit 42. Stop.

  After completing the process in step S54, the operation control unit 48 advances the process to step S55. In Step S51, when it is determined that the state value supplied from the control unit 31 is not “0”, the operation control unit 48 skips Steps S52 to S54 and proceeds to Step S55. Furthermore, when it is determined in step S52 that the notification information provision process (the first notification information provision process or the second notification information provision process) and the low-speed communication process are not performed, the operation control unit 48 performs step S53. And the process of step S54 is abbreviate | omitted and a process is advanced to step S55.

  In step S55, the operation control unit 48 determines whether or not the state value supplied from the control unit 31 is “1”. If it is determined that the state value is “1”, the process proceeds to step S56. Proceed.

  When the status value is “1”, the operation control unit 48 controls each unit of the monitoring device 10 so that the monitoring device 10 does not perform the notification information control process that is a process related to the provision of the notification information. Controls each part of the monitoring device 10 so as to perform low-speed wireless communication with the main control device 20.

  In step S <b> 56, the operation control unit 48 determines whether or not the notification information providing process (first notification information providing process or second notification information providing process) is being executed in each unit of the monitoring device 10. If it is determined that the notification information providing process is being executed, the operation control unit 48 terminates the notification information providing process in step S57, and controls the power supply unit 47 in step S58 to control the imaging unit 61 and the sound collecting unit. The supply of power to the unit 62, the notification information processing unit 44, and the high-speed communication unit 45 is stopped.

  After completing the process in step S58, the operation control unit 48 advances the process to step S59. If it is determined in step S56 that the notification information provision process (the first notification information provision process or the second notification information provision process) has not been executed, the operation control unit 48 performs steps S57 and S58. The process is omitted and the process proceeds to step S59.

  In step S59, the operation control unit 48 determines whether or not the low-speed communication process is being executed, and when it is determined that the low-speed communication unit 42 is not supplied with power and the low-speed communication process is not being executed. The process proceeds to step S60, the power supply unit 47 is controlled to start supplying power to the low-speed communication unit 42, and in step S61, the low-speed communication unit 42 is controlled to start the low-speed communication process, as shown in FIG. The process proceeds to step S71.

  When the operation control unit 48 determines in step S55 that the state value supplied from the control unit 31 is not “1”, or in step S59, the operation control unit 48 determines that the low-speed communication process is being performed. The process proceeds to step S71 of FIG.

  In step S71 of FIG. 11, the operation control unit 48 determines whether or not the state value supplied from the control unit 31 is “2”, and if it is determined that the state value is “2”, step S72 is performed. Proceed with the process.

  When the state value is “2”, the operation control unit 48 controls each unit of the monitoring device 10 so that the monitoring device 10 supplies only the audio data as the notification information to the main control device 20, and the monitoring device 10. Controls each part of the monitoring device 10 so as to perform low-speed wireless communication with the main control device 20.

  In step S <b> 72, the operation control unit 48 determines whether or not a second notification information providing process for supplying image data and audio data as notification information to the main control device 20 is executed in each unit of the monitoring device 10. . If it is determined that the second notification information providing process is being executed, the operation control unit 48 ends the second notification information providing process in step S73, and controls the power supply unit 47 in step S74. The supply of power to the imaging unit 61 is stopped.

  After completing the process in step S74, the operation control unit 48 advances the process to step S75. If it is determined in step S72 that the second notification information provision process has not been executed, the operation control unit 48 omits the processes in steps S73 and S74 and advances the process to step S75.

  In step S75, the operation control unit 48 determines whether or not the first notification information providing process for supplying only the audio data as the notification information to the main control device 20 is being executed, and the power is supplied to the sound collection unit 62 and the like. If it is determined that the first notification information providing process is not executed, the process proceeds to step S76, and the power supply unit 47 is controlled to collect the sound collecting unit 62, the notification information processing unit 44, In addition, the supply of power to the high-speed communication unit 45 is started, and in step S77, these units are controlled to start the first notification information providing process.

  After completing the process in step S77, the operation control unit 48 advances the process to step S78. If it is determined in step S75 that the first notification information providing process is being executed, the operation control unit 48 omits the processes in steps S76 and S77 and advances the process to step S78.

  In step S78, the operation control unit 48 determines whether or not the low-speed communication process is being executed, and when it is determined that the low-speed communication unit 42 is not supplied with power and the low-speed communication process is not being executed. In step S79, the power supply unit 47 is controlled to start supplying power to the low-speed communication unit 42. In step S80, the low-speed communication unit 42 is controlled to start low-speed communication processing. The process proceeds to step S91.

  When the operation control unit 48 determines in step S71 that the state value supplied from the control unit 31 is not “2”, or in step S78, the operation control unit 48 determines that the low-speed communication process is being executed. The process proceeds to 12 step S91.

  In step S91 of FIG. 12, the operation control unit 48 determines whether or not the state value supplied from the control unit 31 is “3”, and if it is determined that the state value is “3”, step S92 is performed. Proceed with the process.

  When the status value is “3”, the operation control unit 48 supplies the control data to the main control device 20 as notification information by the monitoring device 10 through high-speed wireless communication, and acquires control information through low-speed wireless communication. And each part is controlled so that the sensor output is supplied to the main controller 20.

  In step S <b> 92, the operation control unit 48 determines whether or not the first notification information providing process is being executed in each unit of the monitoring device 10. When it determines with the 1st notification information provision process being performed, the operation control part 48 complete | finishes a 1st notification information provision process in step S93.

  The operation control unit 48 that has finished the process of step S93 advances the process to step S94. If it is determined in step S92 that the first notification information providing process has not been executed, the operation control unit 48 skips the process in step S93 and advances the process to step S94.

  In step S94, the operation control unit 48 determines whether or not the second notification information providing process is being executed, the power is not supplied to the imaging unit 61, and the second notification information providing process is executed. If not, the process proceeds to step S95 and the power supply unit 47 is controlled to start supplying power to the imaging unit 61, the sound collection unit 62, the notification information processing unit 44, and the high-speed communication unit 45. In step S96, the respective units are controlled to start the second notification information providing process.

  After completing the process in step S96, the operation control unit 48 advances the process to step S97. If it is determined in step S94 that the second notification information providing process is being executed, the operation control unit 48 omits the processes in steps S95 and S96 and advances the process to step S97.

  In step S97, the operation control unit 48 determines whether or not the low-speed communication process is being executed, and if it is determined that the low-speed communication unit 42 is not supplied with power and the low-speed communication process is not being executed. The process proceeds to step S98, the power supply unit 47 is controlled to start supplying power to the low-speed communication unit 42. In step S99, the low-speed communication unit 42 is controlled to start the low-speed communication process, and the process proceeds to step S100. Proceed with the process.

  If the operation control unit 48 determines in step S91 that the state value supplied from the control unit 31 is not “3”, or if it determines in step S97 that the low-speed communication process is being performed, The process proceeds to S100.

  In step S100, the operation control unit 48 determines whether or not to end the operation control process. If it is determined that the operation control process is not ended, the operation control unit 48 returns the process to step S51 in FIG. 10 and repeats the subsequent processes. That is, the operation control unit 48 controls the operation of each unit based on the state value supplied from the control unit 31 by repeating the processing from step S51 in FIG. 10 to step S100 in FIG.

  In step S100 of FIG. 12, the operation control process is terminated in order to turn off the monitoring device 10 or shift to the suspend state based on a user input or the like input via an input unit (not shown). When it determines with it, the operation control part 48 advances a process to step S101, performs an end process, and complete | finishes an operation control process.

  As described above, when the operation control unit 48 executes the operation control process, the control unit 31 controls the operation of each unit by supplying the operation control unit 48 with a state value indicating the state of the infant 84, The contents of the notification information supplied to the main control device 10 can be determined, control information acquisition, sensor output supply, and the like can be controlled. That is, since the monitoring device 10 can supply the minimum necessary information to the main control device 20 based on the state of the infant 84 to be monitored, the power consumption can be reduced, and the battery can be installed. Even when driven as a power source, it can operate more efficiently, such as being able to operate for a longer time.

  Next, the first notification information providing process for supplying only the audio data as the notification information to the main control device 20 will be described with reference to the flowchart of FIG.

  First, the sound collection unit 62 included in the notification information acquisition unit 43 of the monitoring device 10 is controlled by the operation control unit 48 in step S121, and there is an infant 84 to be monitored using the microphone 81 and the microphone 82. Sound collection processing that collects the sound of the sound space, converts the sound into an electrical signal (sound signal), amplifies the sound signal, and performs A / D (Analog / Digital) conversion to generate sound data Then, the audio data is supplied to the notification information processing unit 44, and the process proceeds to step S122.

  In step S122, the notification information processing unit 44 is controlled by the operation control unit 48, performs filtering processing, correction processing, and the like on the supplied audio data, generates notification information for transmission, and transmits the notification information at high speed. The data is supplied to the communication unit 45, and the process proceeds to step S123. In step S <b> 123, the high speed communication unit 45 is controlled by the operation control unit 48 to perform high speed wireless communication with the main control device 20 via the antenna 12, and transmits the supplied notification information to the main control device 20. The high speed communication unit 45 that has transmitted the notification information advances the process to step S124.

  In step S124, the sound collection unit 62, the notification information processing unit 44, and the high-speed communication unit 45 are controlled by the operation control unit 48 to determine whether or not to end the first notification information providing process. If it is determined, the process returns to step S121, and the subsequent processes are repeated. When the operation control unit 48 determines that the first notification information providing process is to be terminated, the sound collection unit 62, the notification information processing unit 44, and the high-speed communication unit 45 advance the process to step S125, A termination process is performed, and the first notification information provision process is terminated.

  As described above, each of the sound collection unit 62, the notification information processing unit 44, and the high-speed communication unit 45 performs the first notification information providing process and supplies the notification information including only the audio data to the main control device 20.

  Next, the second notification information providing process for supplying the audio data and the image data as notification information to the main control device 20 will be described with reference to the flowchart of FIG.

  First, the imaging unit 61 included in the notification information acquisition unit 43 of the monitoring device 10 is controlled by the operation control unit 48 in step S141, and uses the camera 83 to capture an image of the infant 84 to be monitored and obtain image data. The obtained imaging process is performed, the incident light is photoelectrically converted by a photoelectric conversion element such as a CCD built in the camera 83, an image signal is generated, the image signal is amplified, and A / D (Analog / Digital) conversion is performed. Then, the image data is generated, the image data is supplied to the notification information processing unit 44, and the process proceeds to step S142.

  The sound collection unit 62 included in the notification information acquisition unit 43 of the monitoring apparatus 10 is controlled by the operation control unit 48 in step S142, and uses the microphone 81 and the microphone 82 in the space where the infant 84 to be monitored exists. Performs sound collection processing to collect sound, converts the sound into an electrical signal (sound signal), amplifies the sound signal, A / D converts, etc. to generate sound data, and notifies the sound data While supplying to the process part 44, a process is advanced to step S143.

  In step S143, the notification information processing unit 44 is controlled by the operation control unit 48, performs filtering processing, correction processing, and the like on the supplied image data and audio data, generates notification information for transmission, and notifies the notification. Information is supplied to the high-speed communication unit 45, and the process proceeds to step S144. In step S <b> 144, the high speed communication unit 45 is controlled by the operation control unit 48 to perform high speed wireless communication with the main control device 20 via the antenna 12, and transmits the supplied notification information to the main control device 20. The high speed communication unit 45 that has transmitted the notification information advances the process to step S145.

  In step S145, the imaging unit 61, the sound collection unit 62, the notification information processing unit 44, and the high-speed communication unit 45 determine whether or not to end the second notification information provision process under the control of the operation control unit 48. If it is determined not to end, the process returns to step S141, and the subsequent processes are repeated. In addition, when it is determined by the operation control unit 48 to end the second notification information providing process, the imaging unit 61, the sound collection unit 62, the notification information processing unit 44, and the high-speed communication unit 45 go to step S 146. The process is advanced, an end process is performed, and the second notification information providing process is ended.

  As described above, each of the imaging unit 61, the sound collecting unit 62, the notification information processing unit 44, and the high-speed communication unit 45 performs the second notification information providing process, and performs main control on the notification information including image data and audio data. Supply to device 20.

  Next, low-speed communication processing for supplying sensor output to the main control device 20 and acquiring control information supplied from the main control device 20 will be described with reference to the flowchart of FIG.

  First, in step S161, the low-speed communication unit 42 executes communication connection processing with the main control device 20, and establishes wireless communication with the main control device 20. In step S162, the low speed communication unit 42 determines whether or not the wireless communication with the main controller 20 is established and the connection is completed by the communication connection process in step S161. Proceed with the process.

  In step S163, the low-speed communication unit 42 determines whether or not the control information supplied from the main controller 20 by low-speed wireless communication has been received via the antenna 11, and if it is determined that the control information has been received, the process proceeds to step S164. The process proceeds and the received control information is supplied to the control unit 31. When the process of step S164 ends, the low-speed communication unit 42 advances the process to step S165. If it is determined in step S163 that control information has not been received, the low-speed communication unit 42 omits the process in step S164 and advances the process to step S165.

  In step S165, the low-speed communication unit 42 determines whether or not the sensor output is supplied from the control unit 31. If it is determined that the sensor output is supplied, the process proceeds to step S166, and the supplied sensor output is transmitted to the antenna 11. Via the low-speed wireless communication. When the process of step S166 is completed, the low-speed communication unit 42 advances the process to step S167. If it is determined in step S165 that the sensor output is not supplied, the low-speed communication unit 42 omits the process in step S166 and proceeds to step S167.

  In step S167, the low-speed communication unit 42 determines whether or not to end the low-speed communication process. If it is determined that the low-speed communication process does not end, the process returns to step S163, and the subsequent processes are repeated. If it is determined in step S167 that the low-speed communication process is to be terminated, for example, under the control of the operation control unit 48, the low-speed communication unit 42 proceeds to step S168, performs the end process, and then performs low-speed communication. The process ends.

  If it is determined in step S162 that communication with the main control device 20 has failed to be established and the main control device 20 is not connected, the low-speed communication unit 42 proceeds to step S169 and performs error processing. After the end process is performed in step S168, the low-speed communication process is ended.

  As described above, the low-speed communication unit 42 supplies the sensor output to the main control device 20 or acquires the control information supplied from the main control device 20.

  As described above, the main controller 20 to which the notification information and the like are supplied performs various processes such as presenting the supplied notification information to the user.

  Next, the operation of main controller 20 will be described.

  The control unit 101 of the main control device 20 is based on the user input input via the input unit 141, the sensor output supplied from the monitoring device 10 of FIG. To control the operation.

  When notification information such as audio data or image data is supplied from the monitoring device 10, the high-speed communication unit 112 acquires the notification information via the antenna 22 and stores it in a built-in cache memory (not shown) or the like. The high-speed communication unit 112 supplies the held notification information to the notification information storage unit 114, the composition processing unit 116, and the selector 117 for each predetermined amount of data, for example, and extracts the notification information from the notification information. The audio data is supplied to the filter bank 113.

  When the sensor output is supplied from the monitoring device 10, the low-speed communication unit 111 acquires the sensor output via the antenna 21 and stores it in a built-in cache memory (not shown) or the like. Then, the low speed communication unit 111 supplies the held sensor output to the control unit 101 for each predetermined amount of data, for example.

  Each unit of the main control device 20 controls the reception processing of the notification information and sensor output supplied from the monitoring device 10 by executing the reception control processing. The reception control process will be described with reference to the flowchart of FIG.

  First, the low speed communication unit 111 and the high speed communication unit 112 of the main controller 20 are controlled by the control unit 101 in step S181 to start receiving notification information and sensor output.

  In step S182, the high-speed communication unit 112 determines whether or not the notification information transmitted from the monitoring device 10 has been received. If it is determined that the notification information has been received via the antenna 22, the process proceeds to step S183. The received notification information is temporarily held, and in step S184, audio data is extracted from the held notification information each time a predetermined timing or a predetermined amount of data is held, and the filter is filtered. Supply to the bang 113. In step S185, the high-speed communication unit 112 stores the notification information (including voice data) stored therein at a predetermined timing or a predetermined amount of data. 116 and the selector 117 and the like. The high speed communication unit 112 that has supplied the notification information proceeds to step S186. In step S182, the high speed communication unit 112 that has determined that the notification information has not been received omits steps S183 to S185, and proceeds to step S186.

  In step S186, the low-speed communication unit 111 determines whether or not the sensor output transmitted from the monitoring device 10 has been received. If it is determined that the sensor output has been received via the antenna 21, the process proceeds to step S187. The received sensor output is temporarily held, and at step S188, the sensor output is held every time a predetermined timing or a predetermined amount of data is held, or based on an instruction from the user or the like. The sensor output is supplied to the control unit 101. The low speed communication unit 111 that has supplied the sensor output proceeds to step S189. In step S186, the low-speed communication unit 112 that has determined that the sensor output has not been received omits steps S187 and S188, and proceeds to step S189.

  In step S189, the control unit 101 determines whether or not to end the reception control process. If it is determined that the reception control process is not ended, the control unit 101 returns the process to step S182 and repeats the subsequent processes. If it is determined in step S189 that the reception control process is to be terminated, the control unit 101 executes the termination process in step S190, for example, controls the low-speed communication unit 111 and the high-speed communication unit 112 to output a sensor output or notification. The reception control process is terminated, for example, by terminating the reception of information.

  As described above, main controller 20 receives notification information and sensor output supplied from monitoring device 10.

  As described above, the filter bank 113 that has acquired the audio data supplied from the high-speed communication unit 112 performs low-pass filtering on the audio data using a Wavelet transform, a FIR (Finite Impulse Response) filter, or an IIR (Infinite Impulse Response) filter. Filter processing is performed using a filter, or noise component removal processing is performed. For example, the filter bang 113 extracts only a signal in a frequency band (for example, several hundred Hz to several thousand Hz) including a cry from the acquired voice data by performing filter processing. Note that the filter bang 113 may perform filter processing using a high-pass filter or a band-pass filter together. The noise component removal processing by the filter bank 113 will be described later. In addition, the filter bank 113 thins the data amount of the notification information as necessary, for example, by performing a thinning process so that the data amount is 1/16 of the audio data having a sampling frequency of 24 KHz. You may make it reduce the processing amount in a back | latter stage.

  Next, for example, the noise component removal processing executed in the filter bank 113 in step S255 of FIG. 18 will be specifically described. In the following description, “removal” and “reduction” are used as synonyms.

  For example, it is assumed that an infant 84 is located as shown in FIG. 17 and another person 85 is speaking in the vicinity of the infant 84. Although the feature amount of the crying voice has been described above, the waveform of the audio signal (audio data) supplied to the filter bank 113 is usually as shown in FIG. 18 when there is no external noise. Then, the audio data is processed in the filter bank 113 and converted into an absolute value as shown in FIG. Each part of the filter bank 113 performs processing on the audio signal for calculation as shown in FIG.

  In FIG. 17, the waveform (cry) output from the infant 84 reaches the microphone 81 and then reaches the microphone 82 with a delay amount τ. In the following, the process of correcting the waveform of the microphone 82 in the microphone 81 by the delay amount τ and aligning the phases of the waveforms with each other is referred to as delay amount correction.

  In the noise component removal processing unit 162 shown in FIG. 5, the delay amount determining unit 171 determines the delay amount τ shown in FIG. As a method thereof, for example, the parameter may be released to the user (the user can input information regarding the delay amount τ). For example, a touch panel is superimposed on the monitor 151 of the notification information output unit 143, and the user designates a part where the face of the infant 84 is present on the screen of the monitor 151 based on the image of the infant 84 displayed on the monitor 151. Then, the position is detected by the touch panel as a user input, and the control unit 101 and the delay amount determination unit 171 can calculate and specify the position of the infant 84 based on the user input. The delay amount τ may be calculated in advance from the relationship between the position 82 and the position of the infant 84.

  In addition, the noise component removal processing unit 162 applies feedback to the delay amount determination unit 171 from the output side so that the delay amount determination unit 171 determines a delay amount that can best extract the cry of the infant 84 after processing. Further, based on the sound output from the speaker 152 of the notification information output unit 143, information for determining the delay amount τ that allows the user to hear the infant 84 most clearly is input via the input unit 141. The delay amount determination unit 171 may determine the delay amount τ based on the user input.

  Furthermore, the delay amount determination unit 171 may calculate the delay amount τ based on the correlation coefficient relationship between the audio signal obtained by the microphone 81 and the audio signal obtained by the microphone 82. This is because the amount of delay between two microphones is corrected, and the point where the correlation coefficient between the audio signal obtained at the microphone 81 and the audio signal obtained at the microphone 82 takes the maximum value is the delay amount τ. It is a method.

  The above delay amount τ is also generated in the same way for the voice uttered by another person 85 as a noise source, and a delay amount is generated between the two microphones from any direction.

  In FIG. 20, when the microphone 81 is used as a reference, the waveform from the infant 84 arrives at the microphone 82 later than τ1 after reaching the microphone 81, and the waveform from the other person 85, which is a noise source, reaches the microphone 81. The microphone 82 is reached by τ2 earlier than the arrival. These two waveforms (waveform 211 and waveform 212) in the reference microphone 81 are respectively expressed as x (t ′) (waveform 212) or y (t ′) (waveform 211) based on a certain reference time t ′. Then, the waveform (waveform 221) from the infant 84 in the microphone 82 can be represented by x (t ′ + τ1), and the waveform (waveform 222) from another person 85 that is a noise source is y (t′−τ2). ). When the buffer delay device 213 and the buffer 223 shown in FIG. 20 delay-correct these waveforms (the waveform 211 and the waveform 212 and the waveform 221 and the waveform 222) at the reference time t, the waveform 211 is converted into the waveform 214, Waveform 212 is converted to waveform 215, waveform 221 is converted to waveform 224, and waveform 222 is converted to waveform 225. Further, the adder 231 adds (ADD) the waveforms (the waveform 214 and the waveform 215 and the waveform 224 and the waveform 225) as shown in the equation (1), and the subtractor 232 (Waveform 214 and Waveform 215 and Waveform 224 and Waveform 225) are subtracted (SUB) as shown in Expression (2).

ADD = 2x (t) + y (t) + y (t− (τ1 + τ2)) (1)
SUB = y (t) -y (t- (τ1 + τ2)) (2)

  In the following, the term of the variable x is referred to as the term of the infant 84 and the term of the variable y is referred to as the noise source term. In the noise component removal processing unit 162 shown in FIG. 5, the delay correction addition processing unit 172 performs the arithmetic processing of the equation (1) shown above (addition processing executed by the adder 231 in FIG. 20), and delay correction. The subtraction processing unit 173 executes the arithmetic processing of the equation (2) shown above (subtraction processing executed by the subtractor 232 in FIG. 20).

  Here, based on these formulas (1) and (2), the characteristics of the microphone 81 and the microphone 82 which are the two microphones constituting the sound collection unit 62 of FIG. 2 will be described.

  As shown in FIG. 20, after delay correction is performed centering on the audio signal obtained by collecting the sound from the microphone 81 as the reference microphone, the audio signal from the microphone 81 and the audio signal from the microphone 82 are added (addition). The waveform of the crying voice of the infant 84 can be emphasized, and the waveforms of the noise components from the noise source can be attenuated if they are out of phase with each other.

  In addition, after delay correction centered on the audio signal collected by the microphone 81, which is the reference microphone, the audio signal in the microphone 82 is subtracted (subtracted) from the audio signal in the microphone 81. The waveform can theoretically be eliminated, and the waveform of the noise component from the noise source can be left with the phase changed. In addition, although it described as theoretical here, actually, the waveform of the cry of the infant 84 does not disappear completely in consideration of the attenuation term, reflection from the wall or floor, and the like.

  Since the noise source (for example, another person 85) may move or there may be a plurality of noise sources, the microphone is corrected after delay correction centered on the audio signal obtained by collecting the sound from the microphone 81 as the reference microphone. In some cases, the noise component cannot be sufficiently removed by a temporary process in which the audio signal in the microphone 82 is only drawn from the audio signal in 81.

  Hereinafter, a method for reducing noise other than the target (crying voice of the infant 84) by applying such characteristics will be described. In the above formulas (1) and (2), focusing on the noise source term, the addition result ADD and the subtraction result SUB will be described. The phase relationship between the addition result ADD and the subtraction result SUB will be described. If (τ1 + τ2) is within a half wavelength in any frequency band, as shown in FIG. The term is apparently delayed by ¼ wavelength (π / 2) from the noise source term of Equation (1), which is an addition equation.

  In FIG. 21, a waveform 241 is a waveform of a noise source term (y (t)) corresponding to the microphone 81 in the equation (1) which is an addition equation, and a waveform 242 is an equation (1) which is an addition equation. The waveform of the noise source term (y (t− (τ1 + τ2))) corresponding to the microphone 82 in FIG. 6 is a waveform of the waveform 241 and the waveform 242. The waveform 251 is the waveform of the noise source term (y (t)) corresponding to the microphone 81 in the subtraction formula (2), and the waveform 252 is the microphone in the formula (2) which is the subtraction formula. 82 is a waveform of a noise source term (−y (t− (τ1 + τ2))) corresponding to 82, and a waveform 253 is a combined waveform of the waveform 251 and the waveform 252. In FIG. 21, the phase shift is represented by a single frequency which is an element of a noise source in each microphone.

  In FIG. 21, the phase of the waveform 252 (y (t− (τ1 + τ2))) is shifted by a half wavelength (π) from the phase of the waveform 242 (−y (t− (τ1 + τ2))). A waveform 243 is a combined waveform of the waveform 241 and the waveform 242, and the phase thereof is delayed by (τ 1 + τ 2) / 2 with respect to the phase of the waveform 242. Similarly, the waveform 253 is a combined waveform of the waveform 251 and the waveform 252, and the phase thereof is advanced by (τ 1 + τ 2) / 2 with respect to the phase of the waveform 252. Therefore, the phase difference between the waveform 242 and the waveform 252 is half of the phase difference between the waveform 252 and the waveform 242, that is, ¼ wavelength (π / 2).

  Next, processing executed by the moving average processing unit 174 in FIG. 5 will be described.

  The moving average is mainly used for removing a high frequency band, and any number of points or points may be added during the calculation. In the case of the above-described environment, the moving average MA is expressed as the following equation (3).

  At this time, the function of the waveform to be added is f (x), and the number of samples to be added is α.

  In this way, in the moving average of adding the rear side of itself, as shown in FIG. 22, the phase advances apparently by ½ sample of the added sample number α. In FIG. 22, a graph 260 is a graph in which the horizontal axis indicates the number of samples and the vertical axis indicates the amplification degree. A waveform 261 indicates the waveform of the function f (x) described above, and a waveform 262 indicates the moving average MA calculated from the waveform 261. Further, at this time, the waveform of the moving average MA has a different attenuation rate (r in FIG. 22) with respect to the original waveform depending on the frequency from the 1 / α term of Equation (3).

  As described above, the noise source subtraction result SUB is apparently delayed by ¼ wavelength of the frequency compared to the addition result ADD. The phase of the moving average MA advances by ½ of the number of samples α to be added. Using these features, the addition processing unit 175 in FIG. 5 performs the calculation shown in the following equation (4) to generate a removed waveform NR from which the noise component has been removed.

  NR = ADD / 2−R × MA × SUB (4)

  In Expression (4), R represents a removal rate, and represents a rate at which noise components are removed from the audio signal. As one of the characteristics in the above formula (4), it can be seen that the waveform of the cry of the infant 84 as a target remains the original.

  As described above, by setting optimum parameters (moving average addition sample number α and removal rate R) that can remove the band of the sound generated at the noise source, the noise component removal processing unit 162 can perform more noise. Can be reduced.

  For example, when the infant 84 utters a cry, he / she utters a voice (for example, “Gear” etc.) or breathes (eg “——” etc.). Here, “ー ー ー” means the sound when the infant 84 breathes in between the time when the voice is being spoken and the time when the next voice is being spoken (for example, between “Gear” and “Gear”). (Sound of the part that is not speaking). Therefore, since the part (silent part) that does not make a voice is a redundant area and is a part that is not necessary as voice information, the noise component removal processing unit 162 suppresses the noise of the part, A feature amount with less error can be calculated.

  For this purpose, the noise component removal processing unit 162 uses a correlation coefficient as a relationship between audio signals in the two microphones (the microphone 81 and the microphone 82). FIG. 23 is a diagram for explaining this correlation. 23, a table 270 shows the correlation between the waveform of the cry of the infant 84 in the microphone 81 (waveform 271) and the waveform of the cry of the infant 84 in the microphone 82 (waveform 272), and the noise waveform (waveform in the microphone 81). 273) and the noise waveform (waveform 274) in the microphone 82 is shown. In Table 270, focusing on the waveform of the crying voice of the infant 84, the phase of the waveform 271 and the waveform 272 are aligned by correcting the delay amount, and the difference between these waveforms (between the waveform 271 and the waveform 272). Is small, and the correlation between these waveforms is high. At this time, in the noise waveform, since the phase of the noise itself is different as in the waveform 273 and the waveform 274, the correlation between these waveforms is low.

  Therefore, the correlation coefficient processing unit 176 in FIG. 5 calculates the correlation coefficient r of the audio signal obtained in the two microphones after delay amount correction, as shown by the equation (5).

ただし、

However,

In equation (5), variable m1 is an audio signal input from microphone 81, and variable m2 is an audio signal input from microphone 82. The correlation coefficient r is a correlation coefficient for n pairs of numerical values (m1 ₁ , m2 ₁ ), (m1 ₂ , m2 ₂ ),..., (M1 _n , m2 _n ). Furthermore, the variable S ₁ and the variable S ₂ indicate standard deviations, and are expressed as Expression (6) and Expression (7), respectively.

  The corresponding area determination unit 177 in FIG. 5 determines whether noise is mixed in the cry, or whether it is a utterance area or a silence area, using the correlation coefficient calculated as described above. In each case, the attenuation processing unit 178 is controlled to further reduce the noise component included in the audio data.

  Of the audio data, if the crying part is the uttering part and the non-speeching part is the silent area, the corresponding area determination unit 177 is represented by the following formula (10) and Determination processing is performed based on the conditional expression shown in Expression (11). In Expressions (10) and (11), the variable COR indicates a correlation coefficient, and the variable th is a predetermined threshold value. Expression (10) is a conditional expression for the utterance region, and Expression (11) is a conditional expression for specifying a region that is a noise mixed region and a silent region.

COR> th (10)
COR <th (11)
However,
−1 ≦ th ≦ 1 (12)

  That is, as shown in equations (10) and (11), if the correlation coefficient is COR, the utterance region if the correlation coefficient COR is equal to or greater than the threshold th, and the correlation coefficient COR is equal to or less than the threshold th. By setting the noise mixed area and the silent area, it is possible to determine the area in the audio signal.

  If the corresponding region determination unit 177 in FIG. 5 determines that the current audio signal region is a silent region and a mixed region, the attenuation processing unit 178 is completely removed by the processing up to the addition processing unit 175 in FIG. The noise component that has not been reduced is attenuated again by changing the variable R in Equation (4), thereby further reducing the noise component.

  In FIG. 17, the noise can be reduced even if the positional relationship between the infant 84 and the other person 85 as the noise source is reversed.

  As described above, in order to remove the noise component from the audio signal, the noise component removal processing unit 162 of the filter bank 113 performs the noise removal process on the audio signal whose band is limited supplied from the low-pass filter 161. Execute. The noise removal processing by the noise component removal processing unit 162 will be described with reference to the flowchart of FIG.

  First, in step S211, the delay amount determination unit 171 of the noise component removal processing unit 162 determines the delay amount τ for the two input audio signals, and in step S212, delay correction is performed for the two audio signals. After the correction results are supplied to the delay correction addition processing unit 172 and the delay correction subtraction processing unit 173, the process proceeds to step S213. In step S213, the delay correction addition processing unit 172 adds the obtained two correction results as described above, supplies the addition result to the addition processing unit 175, and advances the processing to step S214. In step S214, the delay correction subtraction processing unit 173 subtracts the obtained two correction results as described above, supplies the subtraction result to the moving average processing unit 174, and proceeds to step S215. In step S215, the moving average processing unit 174 calculates the moving average MA based on the supplied subtraction result as described above, supplies the calculation result to the addition processing unit 175, and advances the process to step S216. .

  In step S216, the addition processing unit 175 performs addition processing (subtraction processing) as described above, removes the noise component, supplies the speech signal from which the noise component has been removed to the correlation coefficient processing unit 176, and proceeds to step S217. Proceed with the process. In step S217, the correlation coefficient processing unit 176 calculates a correlation coefficient based on the supplied audio signal, supplies the correlation coefficient together with the audio signal to the corresponding region determination unit 177, and the process proceeds to step S218. Proceed. In step S218, the corresponding area determination unit 177 supplies the supplied audio signal and correlation coefficient to the attenuation processing unit 178, and performs area determination of the supplied audio signal based on the supplied correlation coefficient. . Then, in step S219, the corresponding region determination unit 177 determines whether or not the region is a utterance region. If it is determined that the region is not a utterance region, the corresponding region determination unit 177 controls the attenuation processing unit 178 in step S220 to Execute attenuation processing. When the process of step S220 ends, the attenuation processing unit 178 advances the process to step S221. If it is determined in step S219 that the region is not an utterance region, the corresponding region determination unit 177 skips step S220 and proceeds to step S221.

  In step S221, the noise component removal processing unit 162 determines whether or not to end the noise removal processing. If it is determined that the noise removal processing does not end, the process returns to step S212, and the subsequent processing is repeated and then input. The process of shaking is performed on the audio signal. If it is determined in step S221 that the noise removal process is to be terminated, the noise component removal processing unit 162 proceeds to step S222, performs the termination process, and then terminates the noise removal process.

  By performing the noise removal processing as described above, the noise component removal processing unit 162 suppresses the error in calculating the feature amount of the target audio signal component even in the audio signal including the noise component regardless of the position of the monitoring target. can do. As a result, the monitoring system 1 can provide the target audio signal with a small noise component to the user even when the noise component is included. Furthermore, the noise component removal processing unit 162 can remove the noise component by the same process regardless of the position of the noise component generation source.

  The filter bank 113 supplies the sound data processed as described above to the footstep analysis unit 121 and the sound analysis unit 122. The voice analysis unit 122 supplied with the voice data analyzes the voice data and extracts the feature amount of the cry of the infant 84. As this feature amount, the voice analysis unit 122 detects the ratio of the utterance time per breathing cycle of the cry and the breath inhalation time as the feature amount. Of course, the frequency representing the pitch of the voice may be used as the feature amount, and the probability that the state of the cry can be accurately specified by increasing the number of feature amounts in combination with the feature amount such as the utterance time described above is Get higher. Further, the state of the person, for example, the body temperature and the amount of movement of the person is measured using a temperature sensor, an infrared sensor, a microwave sensor, or the like, and the voice analysis unit 122 performs image processing based on the information. It is also possible to further narrow down the state of crying by performing.

  The sound analysis unit 122 is supplied with sound data having a waveform as shown in FIG. As shown in FIG. 25, in the cry waveform 281, utterance and respiration are periodically repeated. Usually, as shown in the waveform 281, in one cycle of the cry (section C in the figure), the sound pressure is high during the time of utterance (section A in the figure), and the time for inhaling the breath (in the figure). In the section (B), although the sound is somewhat heard, it is often statistically sufficiently lower than the utterance sound pressure. When the voice data is supplied from the filter bank 113, the voice analysis unit 122 executes the voice analysis process to calculate the feature amount such as the utterance time and the breathing time.

  The voice analysis processing by the voice analysis unit 122 will be described with reference to the flowcharts of FIGS. Moreover, it demonstrates with reference to FIG. 29 as needed.

  When the audio data is supplied, the audio analysis unit 122 first calculates an average value of the signal level per unit time in order to detect the start of the cry in step S241, and detects a blank time corresponding to the change of breathing. . For example, it is assumed that audio data having a waveform as shown in FIG. 29 is supplied to the audio analysis unit 122. Note that the numbers surrounded by a round frame shown in FIG. 29 indicate positions and ranges. In the following, these numbers are surrounded by “”. When the audio data having the waveform as shown in FIG. 29 is supplied, the audio analysis unit 122 executes the process of step S241 at the position “1”.

  Then, in the voice analysis unit 122 step S242, it is determined whether or not the average value of the unit time calculated in step S241 is equal to or less than a predetermined first threshold value, and the voice data shown in FIG. It is confirmed whether or not the current position in is located in the range of the breathing time as in the position “1”. When the average value of the unit time is not less than or equal to the first threshold and it is determined that the current position is the utterance time, the voice analysis unit 122 returns the process to step S241 and repeats the subsequent processes.

  In step S242, when it is determined that the average value of the unit time is equal to or less than the first threshold value and the current position is located in the range of the breathing time, such as the position “1”, the voice analysis unit 122 The process proceeds to step S243.

  In step S243, the voice analysis unit 122 confirms the level of the sample of the supplied voice data, and in step S244, whether or not the level of the sample is equal to or higher than the utterance start level that is a predetermined threshold value. Determine. If it is determined that the level of the obtained sample is smaller than the utterance start level and it is still the breathing time, the voice analysis unit 122 returns the process to step S243 and repeats the subsequent processes for the next supplied sample.

  If it is determined in step S244 that the level of the sample is equal to or higher than the start of utterance and the generation time of the cry is started, the voice analysis unit 122 sets the sample (position) as the start of the utterance of the cry and proceeds to step S245. And the management of the length of the feature amount is started by starting the count of the main counter. For example, in the sample at the position “2” of the signal in FIG. 29, the level is a value equal to or higher than the utterance start level, so the voice analysis unit 122 starts counting of the main counter from this position “2”.

  In step S246, the voice analysis unit 122 that has started counting the main counter determines an average sound pressure (level average value) for a sample in a predetermined section in order to determine a second threshold value that is a level threshold value described later. Is calculated. This second threshold value is calculated in proportion to the average sound pressure. For example, the voice analysis unit 122 calculates the average sound pressure (average level value) of the samples in the section “3” that is a predetermined section within the utterance time in the signal of FIG.

  The voice analysis unit 122 that has calculated the average sound pressure advances the processing to step S247 and waits for a predetermined time in order to perform processing on the end portion of the utterance time. For example, the voice analysis unit 122 waits for a predetermined time from the end position “4” of the section “3” in the signal of FIG.

  When waiting for a predetermined time, the voice analysis unit 122 proceeds with the process to step S251 in FIG. At this time, the voice analysis unit 122 also advances the processing to step S271 in FIG. 28 as described later, and executes the processing after step S271 in FIG. 28 in parallel with the processing after step S251 in FIG. .

  In step S251 of FIG. 27, the voice analysis unit 122 starts counting of the sub-counter in order to measure the silent time (breathing time) (count the number of samples in the breathing time). In step S252, the voice analysis unit 122 determines whether the level (value) of the sample is equal to or higher than a second threshold that is a predetermined threshold. As described above, the second threshold value is calculated by multiplying the average sound pressure calculated in step S246 by a predetermined coefficient. If it is determined that the sample level is equal to or greater than the second threshold and the breathing time has ended or that the breathing time has not yet started, the speech analysis unit 122 proceeds to step S253, and if necessary The sub-counter maximum value, which is a variable that holds a value indicating the length of the temporary breathing time, is updated. In step S254, the variable that holds a value indicating the end position of the temporary breathing time is updated as necessary. The first main counter coordinate is updated, the value of the sub-counter is reset in step S255, the process returns to step S252, and the subsequent processes are repeated.

  In other words, when it is determined in step S252 that the current position is not the breathing time, the voice analysis unit 122 compares the value of the subcounter with the value of the stored subcounter maximum value, and determines the value of the subcounter. Is large, the sub-counter maximum value is updated using the value, and the first main counter coordinate is updated using the value of the main counter at that time. Thereafter, the voice analysis unit 122 returns the value of the sub-counter to the initial value, returns the process to step S252, and repeats the subsequent processes.

  If it is determined in step S252 that the sample value is smaller than the second threshold value, the voice analysis unit 122 proceeds to step S256 and determines whether the count of the main counter is completed. When determining that the count of the main counter continues and has not ended, the voice analysis unit 122 returns the process to step S252 and repeats the subsequent processes. That is, if the voice analysis unit 122 determines in step S252 that the current position is the breathing time from the level of the sample value, in step S256, the count of the main counter is ended, and the section for voice analysis is ended. It is determined whether or not.

  If it is determined in step S256 that the count of the main counter has ended, the voice analysis unit 122 proceeds to step S257, ends the count of the sub counter, and acquires the value of the sub counter maximum value in step S258. In step S259, the first main counter coordinate is acquired. In step S260, the start position of the silent time (breathing time) is calculated based on the acquired maximum value of the sub-counter and the first main counter coordinate. Temporary silence time information is created based on these values.

  For example, in FIG. 29, the voice analysis unit 122 calculates the length of the range “5” indicating the temporary silence time (breathing time) by using the sub-counter, and uses the value of the main counter at that time to calculate the temporary time. The position “6” indicating the end position of the breathing time is obtained, the position “7” indicating the start position of the provisional breathing time is obtained therefrom, and provisional silence time information is created using them.

  The voice analysis unit 122 that has created the temporary silence time information advances the process to step S277 in FIG.

  In addition, as described above, the voice analysis unit 122 performs step S271 in FIG. 28 in order to detect the start of the next cry in parallel with the execution of the processes in steps S251 to S260 shown in FIG. Thru | or the process of step S276 is performed. In steps S271 to S274, the voice analysis unit 122 performs the same processing as that in steps S241 to S244 shown in FIG. 26, and detects the start of the next cry. For example, the voice analysis unit 122 performs the processing of step S271 and step S272 on the signal of FIG. 29, calculates the average value of unit time, and compares it with the first threshold value, so that the current position is respired. If it is confirmed that the position is “8” within the time range, the processing of steps S273 and S274 is performed, and the generation start position “9” is detected by comparing the sample level with the utterance start level.

  If it is determined in step S274 that the level of the sample is greater than or equal to the start of utterance, the voice analysis unit 122 proceeds to step S275 and holds the current main counter value as the second main counter coordinate.

  In step S276, the voice analysis unit 122 determines whether or not the counting of the main counter is completed. When it is determined that the counting is not completed, the process returns to step S271, and the subsequent processes are repeated. If it is determined in step S276 that the main counter has been counted, the voice analysis unit 122 proceeds to step S277.

  In step S277, the voice analysis unit 122 performs provisional silence time information created by the processing in steps S251 to S260 in FIG. 27 and the second main counter detected by the processing in steps S271 to S276 in FIG. Using the coordinates, error information, which is information regarding the error component included in the temporary silence time information, is created, and the error information is subtracted from the temporary silence time information to create the silence time information.

  For example, in FIG. 29, the voice analysis unit 122 subtracts the value of the second main counter coordinate from the value of the first main counter coordinate, so that the provisional silence time end position “9” from the voice start position “9”. The length up to 6 is calculated, and the length is subtracted from the length of the range “5”, which is the “temporary silence time”, as the “error”, thereby “error reduction silence time” (that is, silence time information). Create

  The voice analysis unit 122 that has created the silent time information ends the voice analysis process. By performing the processing as described above, the lengths of the ranges “A”, “B”, and “C” shown in FIG. 25, that is, the feature amount of the cry can be obtained.

  By performing the processing as described above, the speech analysis unit 122 can analyze the speech without performing complicated processing and obtain the feature amount, and thus calculate the feature amount more quickly and more easily. be able to. Thereby, the control apparatus 20 can operate | move more efficiently.

  When a predetermined action occurs in the main control device 20, the feature amount is calculated by the voice analysis unit 122 every time data is input as described above. This feature value is obtained by quantizing the characteristics of the cry and is distributed in an arbitrary space when applied to a large number of data. FIG. 30 shows a distribution diagram of input feature values in the feature value space created using the feature values. In FIG. 30, as described above, the feature amount space 310 is a feature amount space created based on the feature amount, and the vertical axis indicates the respiration ratio and the horizontal axis indicates the crying cycle (number of samples).

  Each feature amount data indicated by dots in the feature amount space 310 shown in FIG. 30 is obtained by plotting the feature amount data calculated as described above in the feature amount space 310. The ranges 310 to 314 are obtained by classifying the feature amount data for each state of the infant 84 set by the parent (user) of the infant 84 who heard the cry of the infant 84 corresponding to each feature amount data. is there. Here, the state indicates an intermediate state such as a calm state where breathing is calm, a state where there is a relatively intense danger of breathing, or a closeness or danger. For example, in FIG. 30, a range 311 and a range 312 indicate ranges that are classified into a sweet state, a range 313 indicates a range that is classified as an intermediate state between sweet and dangerous, and a range 314 is , Indicates a range classified as a dangerous state.

  In the above description, the case where the feature amount data is classified into the three states of sweet, intermediate, and dangerous has been described. However, the present invention is not limited to this, and the content of the state to be classified may be any type. Any number of states may be classified. For example, since the user knows various states based on the daily parent-child relationship, it is possible to give qualitative values such as hungry, changing diapers, holding children, or selfishness. Good.

  As described above, the user operates the input unit 141 to classify each feature amount data distributed in the feature amount space 310 in advance according to the state of the infant 84 and assign a qualitative value. In addition, the user can change the setting once classified as appropriate by operating the input unit 141. Of course, the number of feature data used when a user classifies states may be any number, and the number of users who classify states may be two or more.

  Thus, in the feature amount space 310, qualitative values obtained from the user are associated with the feature amount data of the cry. Main controller 20 can divide feature amount space 310 into qualitative value regions (for example, range 311 to range 314 in the case of FIG. 30) by previously enclosing feature amount data giving the same qualitative value in this way. . As a method of dividing the qualitative value region, the division may be performed in consideration of the ratio of the evaluation values.

  An example of dividing the feature amount space 310 into qualitative value regions is shown in FIG. In FIG. 31, a feature amount table 321 is a table of feature amount spaces, and each qualitative value includes a feature amount space stored in the LUT storage unit 131 as a feature amount table, as will be described later. Digitized and numbered. The numerical value is used as an evaluation value, and the value and resolution are arbitrary. In FIG. 31, as shown in the sample 322 for the feature amount table 321, the evaluation value 3 is associated with a dangerous state. The evaluation value 2 is associated with an intermediate state, the evaluation value 1 is associated with a sweet state, and the evaluation value 0 is associated with no input.

  In FIG. 31, a feature quantity table 321 is a table in which a feature quantity space is divided for each unit band (block) of feature quantities having a predetermined size. Of course, the method of dividing the feature amount space may be other than this. For example, the number of feature amount spaces may be any number, the sizes of the blocks may be different from each other, The shape may be any shape. Further, the range of the area of each block may be variable or fixed.

  FIG. 32 shows an example in which each state is digitized so that each block of the feature amount table shown in FIG. 31 is held in the LUT storage unit 131. In FIG. 32, a feature value table 325 is obtained by quantifying the evaluation value of each block of the feature value table 321 shown in FIG. In the LUT storage unit 131, a feature amount table 325 as shown in FIG. 32 is stored as a reference table.

  Note that the LUT storage unit 131 stores a plurality of feature quantity tables 325 having different contents as shown in FIG. 33, a table list 331 is a list of a plurality of feature quantity tables 325 as shown in FIG. In this table list 331, each feature quantity table is managed by being assigned a table number, and the state value of each block is shown for each feature quantity table.

  The table list 331 records all user input information, and the control unit 101 adds the information every time the user inputs information (updates settings) via the input unit 141. Then, the table list 331 stored in the LUT storage unit 131 is updated. Further, the operation frequency of the input unit 141 (that is, the number of times the table list 331 has been updated) and the like are further updated in the table list 331 by the control unit 101 as needed. The table list 331 can describe various information in this way. For example, information such as input time and other sensor feature amounts can be linked to classification of input information as other information. .

  As described above, the voice analysis unit 122 extracts the feature amount of the cry from the voice data supplied via the filter bank 113. The qualitative value determination unit 123 prepares the state value of the infant 84 corresponding to the feature amount data supplied from the voice analysis unit 122 according to the feature amount table prepared in advance as described above and stored in the LUT storage unit 131. The type of cry desired by the user is specified, and the information is supplied to the control unit 101. Based on the information or user input, the control unit 101 determines again whether to output the image data included in the notification information and present it to the user.

  The image data output determination process will be described with reference to the flowchart of FIG.

  First, in step S291, the qualitative value determination unit 123 obtains a feature amount table from the LUT storage unit 131 via the control unit 101, refers to the feature amount table in step S292, and corresponds to the feature amount data. Determine the value. In step S293, the control unit 101 controls the input unit 141 to determine whether or not a user input specifying whether or not to output image data has been received. If it is determined that the user input has been received, the control unit 101 proceeds to step S294. The process proceeds to determine whether to output image data based on the input user input, and the process proceeds to step S296. If the control unit 101 determines in step S293 that no user input has been received, the control unit 101 proceeds to step S295 and determines whether to output image data based on the state value supplied from the qualitative value determination unit 123. Is determined, and the process proceeds to step S296.

  In step S296, the control unit 101 determines whether or not to output image data based on the processing result in step S294 or the processing result in step S295. If it is determined to output the image data, the control unit 101 proceeds to step S297. Each unit is controlled to output the image data, and the image data output determination process is terminated.

  If it is determined in step S296 that image data is not output, the control unit 101 controls each unit not to output image data and ends the image data output determination process.

  When the notification information is being presented to the user, the footstep analysis unit 121 analyzes the supplied voice data and detects the footstep sound of the user registered in advance, so that the user can install the monitoring device 10, That is, it is determined whether or not it exists at a position close to the infant 84. For example, when the footstep analysis unit 121 detects a user's footsteps from the supplied voice data and determines that the user is located near the infant 84, the footstep analysis unit 121 supplies the information to the control unit 101. Based on the information, the control unit 101 controls the operation of the notification information output unit 143 so as not to output the notification information.

  By doing in this way, main controller 20 controls presentation of notification information to the user based on not only the conditions in monitoring apparatus 10 on the transmission side but also the conditions in main control apparatus 20 on the reception side. Can do.

  The footstep determination process will be described with reference to the flowchart of FIG.

  In step S <b> 311, the footstep analysis unit 121 performs footstep analysis processing on the supplied voice data and supplies the analysis result to the control unit 101. When acquiring the analysis result, the control unit 101 determines whether or not the user is located in the vicinity of the monitoring apparatus 10 in step S312, and if it is determined that the user is located in the vicinity, the process proceeds to step S313 and notification information ( For example, each unit such as the selector 117 and the notification information output unit 143 is controlled so as to stop the output of the image data. After stopping outputting the notification information, the control unit 101 advances the process to step S316.

  In Step S312, when it is determined that the user is not located near the monitoring device 10, the control unit 101 proceeds to Step S314 and determines whether or not the output of notification information is stopped. When it is determined that the output of the notification information is stopped, the control unit 101 proceeds to step S315 and controls each unit such as the selector 117 and the notification information output unit 143 so as to start outputting the notification information. Then, the process proceeds to step S316. If it is determined in step S314 that the output of the notification information is not stopped, the control unit 101 omits the process of step S315 and advances the process to step S316.

  In step S316, the control unit 101 determines whether or not to end the footstep determination process. If it is determined not to end, the control unit 101 returns the process to step S311 and repeats the subsequent processes. In step S316, for example, when it is determined to end the footstep determination process based on a user instruction or the like, the control unit 101 advances the process to step S317, performs an end process, and ends the footstep determination process.

  By doing in this way, main controller 20 controls presentation of notification information to the user based on not only the conditions in monitoring apparatus 10 on the transmission side but also the conditions in main control apparatus 20 on the reception side. Can do.

  The control unit 101 also supplies the notification information storage unit 114 with the feature value of the cry uttered by the infant 84 acquired via the low-speed communication unit 111, the analysis result supplied from the qualitative value determination unit 123, and the like. The notification information accumulation unit 114 accumulates the information associated with the notification information (audio data or image data) supplied from the filter bank 113.

  The video accumulation process will be described with reference to the flowchart of FIG.

  In step S331, the control unit 101 acquires the speech analysis result analyzed by the speech analysis unit 122 and the qualitative value determination result determined by the qualitative value determination processing unit 123 from the qualitative value determination processing unit 123, and step S332 2, the sensor output supplied from the monitoring device 10 is acquired via the low-speed communication unit 111 and supplied to the notification information storage unit 114.

  The notification information storage unit 114 is controlled by the control unit 101, and is supplied from the high-speed communication unit 112 in association with the voice analysis result, the qualitative value determination result, and the sensor output supplied from the control unit 101 in step S333. The notified information is recorded, and the video accumulation process is terminated.

  When the notification information is supplied from the high-speed communication unit 112 to the notification information storage unit 114, the control unit 101 executes the above-described video storage processing as necessary.

  The control unit 101 controls the notification information search unit 115 as necessary to search for notification information associated with a specific condition from the notification information stored in the notification information storage unit 114. Let me get it. The notification information search unit 115, which is controlled by the control unit 101 and retrieves the notification information from the notification information storage unit 114, supplies the notification information to the synthesis processing unit 116.

  That is, the control unit 101 controls the notification information storage unit 114, the notification information search unit 115, the synthesis device 116, and the selector 117 based on the table stored in the LUT storage unit 131, thereby Information that has not been transmitted from the monitoring device 10 is supplemented using information supplied in the past and presented to the user.

  For example, the control unit 101 uses the image data included in the notification information transmitted from the monitoring device 10 as the result of the voice analysis unit 122 and the qualitative value determination unit 123 and the sensor output acquired via the low-speed communication unit 111. The feature information of the voice uttered by the infant 84 included is stored in the notification information storage unit 114 as a key.

  When only the voice data is sent from the monitoring device 10 as the notification information, the control unit 101 controls the notification information search unit 115 to check the results of the voice analysis unit 122 and the qualitative value determination unit 123, the sensor output, and the like. As a key, image data having a situation similar to the current situation of the infant 84 is retrieved from the information accumulated in the notification information accumulation unit 114 and acquired and supplied to the synthesizing device 116. Furthermore, the monitoring device 10 controls the synthesis device 116 and the selector 117 to synthesize the image data stored in the notification information storage unit 114 with the audio data supplied from the monitoring device 10 and notify the notification information as necessary. The data is output via the output unit 143.

  In this manner, the control unit 101 stores the notification information supplied from the monitoring device 10 in the notification information storage unit 114 in association with the situation of the infant 84, thereby notifying the notification information not supplied from the monitoring device 10. Part or all of the data is supplemented using past notification information stored in the notification information storage unit 114, and the notification information output unit 143 is controlled to be presented to the user.

  By doing in this way, the main control apparatus 20 can remarkably reduce radio | wireless communication with heavy power consumption, without making a user feel inconvenience. As a result, the main control device 20 can operate the monitoring device 10 more efficiently, such as enabling the monitoring device 10 to operate for a longer period of time, even when the monitoring device 10 is driven using a battery as a power source.

  The image data substitution determination process for performing the above process will be described with reference to the flowchart of FIG.

  First, in step S351, the control unit 101 controls the high-speed communication unit 112 to determine whether or not image data has been supplied. If it is determined that the image data has not been supplied, the control unit 101 proceeds to step S352 and performs audio data It is determined whether or not is supplied. If it is determined that the audio data has been supplied, the control unit 101 proceeds to step S353 to acquire the audio analysis result and the qualitative value determination result supplied from the qualitative value determination unit 123, and notify them of the notification information search unit 115. To supply. The notification information search unit 115 that has acquired the voice analysis result and the qualitative value determination result is controlled by the control unit 101, accesses the notification information storage unit 114 in step S354, and based on the acquired voice analysis result and qualitative value determination result. To search for and obtain past notification information corresponding to them. The notification information search unit 115 that acquired the past notification information supplies the past notification information to the synthesis processing unit 116.

  In step S <b> 355, the composition processing unit 116 is controlled by the control unit 101 and is supplied from the high-speed communication unit 112 as well as past notification information stored in the notification information storage unit 114 acquired from the notification information search unit 115. The current notification information is synthesized and supplied to the selector 117. The selector 117 is controlled by the control unit 101 to supply the combined notification information to the notification information output unit 143, and the process proceeds to step S358.

  If it is determined in step S352 that the audio data is not supplied, the control unit 101 cannot acquire the analysis result or the determination result. Therefore, in step S356, the control unit 101 controls the low-speed communication unit 111 to monitor the monitoring device 10. The sensor output supplied is acquired, and the sensor output is supplied to the notification information search unit 115. In step S357, the notification information search unit 115 supplied with the sensor output accesses the notification information storage unit 114 under the control of the control unit 101, searches past notification information based on the supplied sensor output, get. The notification information search unit 115 that acquired the past notification information supplies the past notification information to the synthesis processing unit 116. The composition processing unit 116 is controlled by the control unit 101 and supplies the past notification information stored in the notification information storage unit 114 acquired from the notification information search unit 115 to the selector 117. The selector 117 is controlled by the control unit 101 to supply the past notification information to the notification information output unit 143, and the process proceeds to step S358.

  In step S358, the notification information output unit 143 supplied with the combined notification information or the past notification information is controlled by the control unit 101 in step S358 and outputs the notification information. The notification information output unit 143 that has output the notification information advances the process to step S359.

  If it is determined in step S351 that image data is supplied, the control unit 101 advances the process to step S359. In step S359, the control unit 101 determines whether or not to end the image data substitution determination process. If it is determined not to end, the control unit 101 returns the process to step S351 and repeats the subsequent processes. If it is determined in step S359 that the image data substitution determination process is to be terminated, the control unit 101 advances the process to step S360, executes the termination process, and ends the image data substitution determination process.

  In addition, the control unit 101 executes simple display processing based on a user instruction or the like, creates simple display information based on a sensor output or the like acquired via the low-speed communication unit 111, and performs a simple display unit 142. To display. The simple display process will be described with reference to the flowchart of FIG.

  The control unit 101 that has executed the simple display process based on a user instruction or the like first controls the low-speed communication unit 111 and acquires the sensor output supplied from the monitoring device 10 in step S401. In step S <b> 402, the control unit 101 that has acquired the sensor output creates simple display data such as a message based on the sensor output, and supplies the simple display data to the simple display unit 142. In step S403, the simple display unit 142 supplied with the simple display data is controlled by the control unit 101 to output the simple display data on the display. The control unit 101 that controls the simple display unit 142 and outputs the simple display data ends the simple display process.

  As described above, main controller 20 simply notifies the user of the status of infant 84 using not only notification information such as image data and audio data but also simple display data such as character data. be able to.

  By doing in this way, the main control apparatus 20 can remarkably reduce radio | wireless communication with heavy power consumption, without making a user feel inconvenience. As a result, the main control device 20 can operate the monitoring device 10 more efficiently, such as enabling the monitoring device 10 to operate for a longer period of time, even when the monitoring device 10 is driven using a battery as a power source.

  Further, the control unit 101 creates a new table (update table) based on a user input or the like supplied via the input unit 141. In that case, the control unit 101 stores the created update table in the LUT storage unit 131 or supplies the update table to the monitoring device 10 via the low-speed communication unit 111. As a result, the main controller 20 can cause the monitoring device 10 to use an arbitrary table.

  The control unit 101 further selects an appropriate table from the tables stored in the LUT storage unit 131 based on the user's instruction and the like, and stores the table number designation information including the table number of the table at a low speed. The data is supplied to the monitoring device 10 via the communication unit 111. As a result, the main controller 20 can specify a table used in the monitoring device 10.

  In addition, when there is an input meaning “important” from the user via the input unit 141, the control unit 101 sets the value of the corresponding part of the table 91 to “3”, and thereafter, the past video is not used. Like that. At that time, the updated table 91 is transmitted to the monitoring apparatus 10 and reflected in the operation of each unit.

  By doing in this way, the main control apparatus 20 supplies control information to the monitoring apparatus 10 via the low-speed communication part 111, when some or all of notification information is not supplied from the monitoring apparatus 10, and notification information Can be retransmitted.

  Conversely, if there is no input meaning “important” via the input unit 141, the control unit 101 may control each unit to reduce the amount of communication, and further eliminate the notification itself. Further, the communication amount may be further reduced.

  Further, the control unit 101 specifies a sensor of the sensor unit 41 used in the monitoring device 10 based on a user instruction or the like.

  As described above, the main control device 20 can reflect the instruction input by the user in the process by supplying the control information to the monitoring device 10 via the low-speed communication unit 111.

  User instruction acceptance processing will be described with reference to the flowcharts of FIGS. 39 and 40.

  The control unit 101 that has input the instruction information and started the user instruction receiving process first determines whether or not a table to be used is designated by the user in step S421, and stores it in the LUT storage unit 131. If it is determined that the use table is designated from the plurality of feature amount tables, the process proceeds to step S422, the setting relating to the use table is updated using the designated table number, and the low-speed communication unit is obtained in step S423. Through 111, table number designation information including the designated table number is supplied to the monitoring apparatus 10.

  The control unit 101 that has finished the process of step S423 advances the process to step S424. If it is determined in step S421 that the table to be used is not specified by the user, the processes in steps S422 and S423 are omitted, and the process proceeds to step S424.

  In step S424, the control unit 101 determines whether or not a new table has been created by the user. If it is determined that the table has been created, the control unit 101 supplies the created new table to the LUT storage unit 131, and step S425 is performed. Proceed with the process. In step S425, the LUT storage unit 131 is controlled by the control unit 101 and stores the supplied new table. In step S426, the control unit 101 that has saved the new table updates the setting of the usage table, sets the newly created table in the usage table, and in step S427, the control unit 101 stores the new table through the low-speed communication unit 111. A new table is supplied to the monitoring apparatus 10 as an update table.

  The control unit 101 that has finished the process of step S427 advances the process to step S431 of FIG. Further, in step S424 in FIG. 39, when it is determined that a new feature amount table has not been created by the user, the control unit 101 omits the processes in steps S425 to S427 and performs the process in step S431 in FIG. Proceed.

  In step S431 in FIG. 40, the control unit 101 determines whether or not the feature amount table has been updated by the user. If it is determined that the feature amount table has been updated, the control unit 101 supplies the update information to the LUT storage unit 131, and the process proceeds to step S432. Proceed with the process. In step S432, the LUT storage unit 131 is controlled by the control unit 101 and stores the supplied update information. In step S433, the control unit 101 that has stored the update information updates the setting of the use table, reflects the update information in the use table, and in step S434, updates the updated table via the low-speed communication unit 111. The update table is supplied to the monitoring device 10.

  The control unit 101 that has finished the process of step S434 advances the process to step S435. If it is determined in step S431 that the feature amount table has not been updated by the user, the control unit 101 skips steps S432 to S434 and proceeds to step S435.

  In step S435, the control unit 101 determines whether or not the sensor used in the monitoring device 10 is specified by the user. If it is determined that the sensor is specified, in step S436, the control unit 101 passes the low-speed communication unit 111. Then, the sensor designation information is supplied to the monitoring device 10, and the user instruction receiving process is terminated. If it is determined in step S435 that the sensor used in the monitoring apparatus 10 is not designated by the user, the control unit 101 omits the process in step S436 and ends the user instruction reception process.

  As described above, the main control device 20 can reflect the instruction input by the user in the process by supplying the control information to the monitoring device 10 via the low-speed communication unit 111.

  In addition, the control part 31 of the monitoring apparatus 10 controls each part based on the control information supplied from the main control apparatus 20 by the above processes. When power is supplied to the monitoring device 10 and power is supplied to the control unit 31, the control unit 31 controls the low-speed communication unit 42 to acquire control information supplied from the main control device 20 and control the control unit 31. Information reflection processing is executed, and each unit is controlled based on the acquired control information. The control information reflection process will be described with reference to the flowchart of FIG.

  First, in step S451, the control unit 31 determines whether or not control information has been acquired via the low-speed communication unit 42. If it is determined that the control information has been acquired, the control unit 31 proceeds to step S452 and acquires the control information. However, it is determined whether or not the feature amount table 181 used in the monitoring apparatus 10 is table number designation information for designating using the table number. When it is determined that the acquired control information is table number specifying information or the acquired control information includes table number specifying information, the control unit 31 advances the process to step S453 and corresponds to the specified table number. The feature amount table is read from the LUT storage unit 46, and the feature amount table is stored in the RAM 32 and set as a use table.

  Control part 31 which ended processing of Step S453 advances processing to Step S454. If it is determined in step S452 that the acquired control information is not table number designation information or the acquired control information does not include table number designation information, the control unit 31 omits the process of step S453. Then, the process proceeds to step S454.

  In step S454, the control unit 31 determines whether or not the acquired control information is an update table updated in the main control device 20. When it is determined that the acquired control information is an update table, or the acquired control information includes an update table, the control unit 31 advances the process to step S455 and stores the supplied update table in the LUT storage unit 46. It is stored in the update LUT storage unit 72, and in step S456, the update table is set in the RAM 32 and set as a use table.

  The control unit 31 that has finished the process of step S456 advances the process to step S457. If it is determined in step S454 that the acquired control information is not an update table, or the acquired control information does not include an update table, the control unit 31 omits the processes in steps S455 and S456, The process proceeds to step S457.

  In step S457, the control unit 31 determines whether or not the acquired control information is sensor designation information that designates a sensor to be operated in the sensor unit 41. When it is determined that the acquired control information is sensor designation information or the acquired control information includes delicate designation information, the control unit 31 proceeds to step S458 to control and operate the sensor unit 41. The sensor is switched to the sensor designated by the sensor designation information.

  The control unit 31 that has finished the process of step S458 advances the process to step S459. If it is determined in step S457 that the acquired control information is not sensor designation information, or the acquired control information does not include sensor designation information, the control unit 31 omits the process of step 458, The process proceeds to S459. Furthermore, when it is determined in step S451 that the control information has not been acquired, the control unit 31 omits steps S452 to S458 and proceeds to step S459.

  In step S459, the control unit 31 determines whether or not to end the control information reflection process. If it is determined not to end, the control unit 31 returns the process to step S451 and repeats the subsequent processes. In step S459, when the control information reflection process is terminated in order to turn off the monitoring apparatus 10 or shift to the suspend state based on a user input or the like input via an input unit (not shown) or the like. When it determines, the control part 31 advances a process to step S460, performs an end process, and complete | finishes a control information reflection process.

  As described above, the control unit 31 reflects the control information. Thereby, main controller 20 can control the operation of monitoring device 10 by supplying control information.

  The main controller 20 shown in FIG. 1 and FIG. 4 may have any size as long as it can be realized. For example, the main controller 20 may be a stationary device installed at a predetermined position. It may be a mobile device portable by the user.

  Moreover, although the example of the internal configuration of the main controller 20 has been described with reference to FIG. 4, the present invention is not limited to this. For example, as shown in FIG. 42, the main controller shown in FIG. You may make it implement | achieve the one part function of 20 as another body.

  In FIG. 42, a remote controller 340 is a device corresponding to the main control device 360 and is a device of a size that can be carried by the user, and includes a low-speed communication unit 341, an antenna 342, an input unit 343, a simple display unit 344, and a monitor 351. And a notification information output unit 345 including a speaker 352, a high-speed communication unit 346, and an antenna 347. The low-speed communication unit 341 of the remote controller 340 performs wireless communication with the low-speed communication unit 111 of the main control device 360 via the antenna 342, and supplies the user input input via the input unit 343 to the main control device 360. . Further, when the low speed communication unit 341 performs wireless communication with the low speed communication unit 111 and acquires the simple display information, the low speed communication unit 341 supplies the simple display unit 344 to display it. The input unit 343 is an input interface similar to the input unit 141 in FIG. 4, receives an input from the user, and supplies it to the low-speed communication unit 341. The simple display unit 344 is a display unit similar to the simple display unit 142 described above, and simply displays a message acquired via the low speed communication unit 341.

  Further, when the high-speed communication unit 346 performs wireless communication with the high-speed communication unit 112 of the main control device 360 via the antenna 347 and acquires notification information, the high-speed communication unit 346 supplies the notification information to the notification information output unit 345 and outputs it. Let The notification information output unit 345 includes a monitor 351 and a speaker 352 as with the notification information output unit 143 shown in FIG. 4, and outputs notification information supplied via the high-speed communication unit 346 using them. To do.

  The main controller 360 is an apparatus in which the input unit 141, the simple display unit 142, and the notification information output unit 143 are configured as a remote controller 340 from the main controller 20 shown in FIG. In the main controller 360, the output of the selector 117 is supplied to the high speed communication unit 112 and is supplied to the remote controller 340 via the high speed communication unit 112.

  When the main control device 360 is such a device, the main control device 360 may be a stationary device, and the user may carry the remote controller 340.

  The present invention can also be applied to an information processing apparatus having a function other than the functions described above. Accordingly, the monitoring device 10 or the main control device 20 described above may of course have functions other than those described above.

  The series of processes described above can be executed by hardware, or can be executed by software as described above. When a series of processing is executed by software, various functions can be executed by installing a computer in which the programs that make up the software are installed in dedicated hardware, or by installing various programs. For example, it is installed from a recording medium or the like into a general-purpose personal computer or the like.

  FIG. 43 is a diagram showing an example of the internal configuration of a personal computer that executes such processing.

  A CPU (Central Processing Unit) 401 of the personal computer 400 executes various processes in accordance with programs stored in the ROM 402. A RAM (Random Access Memory) 403 appropriately stores data and programs necessary for the CPU 401 to execute various processes.

  The CPU 401, ROM 402, and RAM 403 are connected to each other via a bus 404. An input / output interface 410 is also connected to the bus 404.

  The input / output interface 410 is connected to an input unit 411 including a keyboard and a mouse, and outputs a signal input to the input unit 411 to the CPU 401. The input / output interface 410 is also connected to an output unit 412 including a display and a speaker.

  Furthermore, a storage unit 413 configured from a hard disk or the like and a communication unit 414 that performs data communication with other devices via a network such as the Internet are connected to the input / output interface 410. The drive 415 is used when data is read from or written to a removable medium 416 formed of a recording medium such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory.

  As shown in FIG. 43, the recording medium is distributed to provide a program to the user separately from the personal computer, and includes a magnetic disk (including a flexible disk) on which the program is recorded, an optical disk (CD-ROM). (Including Compact Disc-Read Only Memory), DVD (Digital Versatile Disc), a magneto-optical disc (including MD (Mini-Disc) (registered trademark)), or removable media 121 including package media such as semiconductor memory In addition to the above, the ROM 402 storing the program and the hard disk including the storage unit 413 provided to the user in a state of being preinstalled in the computer are included.

  In the present specification, the step of describing the program recorded on the recording medium is not limited to the processing performed in chronological order according to the described order, but is not necessarily performed in chronological order. It also includes processes that are executed individually.

  Further, in this specification, the system represents the entire apparatus constituted by a plurality of apparatuses.

It is a block diagram which shows the structural example of the monitoring system to which this invention is applied. It is a block diagram which shows the detailed structural example of the monitoring apparatus shown by FIG. It is a block diagram which shows the detailed structural example of the notification information acquisition part shown by FIG. It is a block diagram which shows the detailed structural example of the main controller shown by FIG. FIG. 5 is a block diagram illustrating a detailed configuration example of a filter bank illustrated in FIG. 4. It is a flowchart explaining a sensor switching process. It is a flowchart following FIG. 6 explaining a sensor switching process. It is a flowchart explaining a monitoring control process. It is a figure which shows the structural example of the table memorize | stored in the LUT memory | storage part of FIG. It is a flowchart explaining an operation control process. It is a flowchart following FIG. 10 explaining an operation control process. FIG. 12 is a flowchart following FIG. 11 for explaining the operation control process. It is a flowchart explaining a 1st notification information provision process. It is a flowchart explaining a 2nd notification information provision process. It is a flowchart explaining a low-speed communication process. It is a flowchart explaining a reception control process. It is a schematic diagram explaining the amount of delay. It is a figure which shows the example of the waveform of the audio | voice data input into a filter bank. It is a figure which shows the example of the waveform of the audio | voice data converted into the absolute value. It is a figure explaining the principle of delay amount correction. It is a figure explaining the relationship between an addition type and a subtraction type. It is a figure explaining a moving average. It is a figure explaining a correlation. It is a flowchart explaining a noise removal process. It is a figure which shows the example of the waveform of the infant's cry. It is a flowchart explaining an audio | voice analysis process. 27 is a flowchart for explaining the voice analysis process, following FIG. It is a flowchart following FIG. 27 explaining an audio | voice analysis process. It is a figure explaining the example of the mode of an audio | voice analysis process. It is a figure which shows the example of the feature-value data distributed in feature-value space. It is a figure which shows the example of the feature-value table corresponding to feature-value space. It is a figure which shows the example of the digitized feature-value table. It is a figure explaining the example of a table list. It is a flowchart explaining an image data output determination process. It is a flowchart explaining a footstep determination process. It is a flowchart explaining an image | video accumulation | storage process. It is a flowchart explaining an image data substitution determination process. It is a flowchart explaining a simple display process. It is a flowchart explaining a user instruction | indication reception process. It is a flowchart following FIG. 39 explaining a user instruction reception process. It is a flowchart explaining a control information reflection process. It is a figure which shows the structural example of the main controller and remote controller to which this invention is applied. It is a figure which shows the structural example of the personal computer to which this invention is applied.

Explanation of symbols

  1 monitoring system, 10 monitoring device, 20 main control device, 31 control unit, 41 sensor unit, 42 low-speed communication unit, 43 notification information acquisition unit, 44 notification information processing unit, 45 high-speed communication unit, 46 LUT storage unit, 47 power Supply unit, 48 operation control unit, 62 sound collection unit, 81 and 82 microphones, 84 infants, 85 other persons, 101 control unit, 111 low-speed communication unit, 112 high-speed communication unit, 113 filter bank, 114 notification information storage unit, 115 notification information search unit, 116 synthesis processing unit, 117 selector, 121 footstep analysis unit, 122 speech analysis unit, 123 qualitative value determination unit, 131 LUT storage unit, 141 input unit, 142 simple display unit, 143 notification information output unit, 151 Monitor, 152 Speaker, 161 Low-pass fill , 162 Noise component removal processing unit, 171 delay amount determination unit, 172 delay correction addition processing unit, 173 delay correction subtraction processing unit, 174 moving average processing unit, 175 addition processing unit, 176 correlation coefficient processing unit, 177 applicable region determination Unit, 178 attenuation processing unit, 181 feature amount table, 310 feature amount space, 321 feature amount table, 325 feature amount table, 331 table list, 340 remote controller, 341 low speed communication unit, 342 antenna, 343 input unit, 344 simple display Part, 345 notification information output part, 346 high-speed communication part, 347 antenna, 360 main controller, 400 personal computer 416 removable media

Claims (15)

A monitoring system comprising a monitoring device that monitors a monitoring target and a control device that controls the monitoring device,
The monitoring device
Acquiring the image and sound of the monitoring target, and generating notification information for notifying the state of the monitoring target, including the image and the sound;
Supplying the generated notification information to the control device that presents the notification information to a user;
The controller is
Obtaining the notification information supplied from the monitoring device;
Analyzing the monitoring target voice included in the acquired notification information,
Determine the content of the notification information;
When it is determined that the monitoring target image is not included in the notification information, the past corresponding to the analysis result of the monitoring target sound included in the notification information that does not include the monitoring target image. The notification information is searched from a recording unit that records the analysis result of the monitoring target voice in association with the notification information,
A monitoring system for presenting the searched past notification information to the user. The monitoring device
Detecting an environmental change caused by the monitoring target;
Supplying the control device with the detection result of the environmental change by the monitoring target;
The controller is
Obtaining the detection result supplied from the monitoring device;
When it is determined that the image and sound to be monitored are not included in the notification information, the past notification information corresponding to the acquired detection result is recorded in further association with the detection result. Search from the recording section,
The monitoring system according to claim 1, wherein the searched past notification information is presented to the user. The controller is
The user accepts designation of a sensor used by the monitoring device to detect an environmental change by the monitoring target,
Supply the received designation of the sensor as control information to the monitoring device,
The monitoring device
Obtaining the control information supplied from the control device;
The monitoring system according to claim 2, wherein the sensor specified by the control information is selected and used to detect an environmental change due to the monitoring target. The monitoring system according to claim 2, wherein the monitoring device and the control device exchange the notification information through a dedicated line different from other information. The controller is
Accepting the creation or update of table information that associates the voice characteristics of the monitoring target and the status of the monitoring target by the user, or designation of the table information,
Control regarding acquisition of the notification information is performed based on the detection result, using the new table information that is the created table information, the updated table information that is the updated table information, or the designation of the table information as control information. Supply to the monitoring device using the table information for
The monitoring device
Obtaining the control information supplied from the control device;
If control information including the new table information is acquired, store the new table information, and set to use the new table information,
If control information including the update table information is acquired, store the update table information, set to use the update table information,
The monitoring system according to claim 2, wherein when control information including designation of the table information is acquired, the table information designated by the control information is set to be used. A control device for controlling a monitoring device for monitoring a monitoring target;
Information for notifying the status of the monitoring target supplied from the monitoring device, the notification information acquiring means for acquiring notification information including the image and sound of the monitoring target;
Presenting means for presenting the notification information to a user;
Voice analysis means for analyzing the monitoring target voice included in the notification information acquired by the notification information acquisition means;
A recording unit that records the result of analysis by the voice analysis unit in association with the notification information acquired by the notification information acquisition unit;
Notification information determination means for determining the content of the notification information;
When the notification information determination unit determines that the monitoring target image is not included in the notification information, the voice analysis unit includes the monitoring included in the notification information that does not include the monitoring target image. Search means for searching for the past notification information corresponding to the result of analysis of the target speech from the notification information recorded by the recording means, and
The presenting means presents the past notification information retrieved by the retrieval means to the user when the notification information determination means determines that the image to be monitored is not included in the notification information. apparatus. A detection result acquisition means for acquiring a detection result of an environmental change due to the monitoring target from the monitoring device;
The recording means records the detection information associated with the detection result acquired by the detection result acquisition means in association with the notification information;
When it is determined by the notification information determination means that the notification information does not include the image and sound to be monitored,
The search means searches the notification information in the past corresponding to the detection result acquired by the detection result acquisition means from the notification information recorded by the recording means,
The control device according to claim 6, wherein the presenting unit presents the past notification information retrieved by the retrieving unit to the user. Footstep analysis means for analyzing the user's footsteps based on the notification information acquired by the notification information acquisition means;
User position determination means for determining whether or not the user is located in the vicinity of the monitoring target based on the analysis result of the footsteps by the footstep analysis means;
The control device according to claim 6, wherein the presenting unit stops presenting the notification information when the user position determining unit determines that the user is located in the vicinity of the monitoring target. Table information storage means for storing table information for associating the voice characteristics of the monitoring target with the status of the monitoring target;
Using the table information acquired from the table information storage means, further comprising a state determination means for determining the status of the monitoring target from the result of the analysis of the monitoring target voice by the voice analysis means,
The control device according to claim 6, wherein the presenting unit controls whether to present the notification information to the user according to the state of the monitoring target determined by the state determining unit. The apparatus further includes a reception unit that receives the creation or update of the table information by the user, or the specification of the table information used by the state determination unit,
When creation of the table information is accepted by the accepting means,
The table information storage means stores the created table information,
The state determination means is set to use the created table information,
When the update of the table information is received by the receiving means,
The table information storage means updates the stored table information,
The state determination means is set to use the updated table information,
When the specification of the table information is received by the receiving means,
The control device according to claim 9, wherein the state determination unit is configured to use the specified table information. In order to control the acquisition of the notification information based on the detection result of the environmental change by the monitoring target, using the created table information, the updated table information, or the specification of the table information as control information The control device according to claim 10, further comprising control information supply means for supplying the monitoring device with table information. Accepting means for accepting designation of a sensor used by the user for detecting an environmental change by the monitoring target by the monitoring device;
The control device according to claim 6, further comprising: a control information supply unit that supplies the monitoring device with the designation of the sensor received by the reception unit as control information. The control device according to claim 6, wherein the notification information acquisition unit acquires the notification information supplied from the monitoring device via a dedicated line different from other information. A control method of a control device that controls a monitoring device that monitors a monitoring target,
Information for notifying the status of the monitoring target supplied from the monitoring device, the notification information acquiring step for acquiring notification information including the image and sound of the monitoring target;
A voice analysis step of analyzing the monitoring target voice included in the notification information acquired by the processing of the notification information acquisition step;
A notification information determination step of determining the content of the notification information;
When it is determined by the process of the notification information determination step that the monitoring target image is not included in the notification information, the notification information by the voice analysis step process that does not include the monitoring target image The past notification information corresponding to the analysis result of the included voice to be monitored is associated with the notification information acquired by the notification information acquisition step and the analysis result by the processing of the voice analysis step. A search step for searching from the notification information recorded by the recording unit for recording;
A presenting step of presenting the user with the past notification information retrieved by the processing of the retrieval step. In a program that causes a computer to perform processing for controlling a monitoring device that monitors a monitoring target,
Information for notifying the status of the monitoring target supplied from the monitoring device, the notification information acquiring step for acquiring notification information including the image and sound of the monitoring target;
A voice analysis step of analyzing the monitoring target voice included in the notification information acquired by the processing of the notification information acquisition step;
A notification information determination step of determining the content of the notification information;
When it is determined by the process of the notification information determination step that the monitoring target image is not included in the notification information, the notification information by the voice analysis step process that does not include the monitoring target image The past notification information corresponding to the analysis result of the included voice to be monitored is associated with the notification information acquired by the notification information acquisition step and the analysis result by the processing of the voice analysis step. A search step for searching from the notification information recorded by the recording unit for recording;
A program that causes a computer to execute the presenting step of presenting the past notification information retrieved by the processing of the retrieval step to the user .

Images