JP2022155584A - Detection system and space utilization system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a detection system configured to detect an occurrence of a specific event in a predetermined space with high accuracy, and a detection method.

SOLUTION: A detection system comprises at least one computer device, and includes: first information acquisition means for acquiring first information from a predetermined space; second information acquisition means for acquiring second information, which is different from the first information, from the predetermined space; and condition determination means which determines whether the acquired first and second information satisfies a predetermined condition that indicates an occurrence of a predetermined event in the predetermined space. The first information is information on a captured image of the predetermined space. The second information is information on a temperature in the predetermined space. The specific event is a dozing state of a person in the predetermined space, or a sign of doze.

SELECTED DRAWING: Figure 4

COPYRIGHT: (C)2023,JPO&INPIT

Description

The present invention relates to a detection system and detection method capable of detecting the occurrence of a specific event in a predetermined space. The present invention also relates to a space utilization system and a space utilization method that give benefits to users who use a predetermined space.

Conventionally, services for renting or sharing vehicles such as automobiles have been provided. These services have the advantage of being able to use a vehicle at a desired timing without owning a vehicle.

In these car rental and car sharing services, vehicles are used by unspecified users. can no longer be used comfortably. In order to solve such a problem, for example, an information processing system that acquires the intensity of the odor in the interior of the vehicle and outputs information for requesting cleaning of the vehicle when the intensity of the odor is equal to or greater than a threshold is developed. (see, for example, Patent Document 1).

By the way, there are devices other than Japanese Patent Application Laid-Open No. 2002-200318 that can detect the occurrence of a specific event using a sensor. For example, for the purpose of caring for a sick person, an infant, etc., image data of the target person is acquired, odor data around the target person is acquired, and the condition of the target person is acquired based on these image data and odor data. A monitoring system for discriminating levels has been disclosed (see, for example, Patent Document 2).

Although such a system has been disclosed, there are cases where it is not possible to accurately detect the occurrence of a specific event in a predetermined space.

JP 2020-189599 A JP 2013-164684 A

The present invention has been made to solve such problems. That is, a first object of the present invention is to provide a detection system and detection method capable of detecting with high accuracy that a specific event has occurred in a predetermined space.

A second object of the present invention is to provide a novel space utilization system and space utilization method that give benefits to users who use a predetermined space.

The subject of the present invention is
[1] A first information obtaining means comprising at least one computer device for obtaining first information from a predetermined space, and a second information obtaining means for obtaining second information different from the first information from the predetermined space. Detection comprising information acquisition means and condition determination means for determining whether or not the acquired first information and second information satisfy a predetermined condition indicating that a specific event has occurred in a predetermined space. system;
[2] A first condition determination means for determining whether or not the acquired first information satisfies a predetermined first condition; a second condition determining means for determining whether or not the The detection system according to [1] above, which determines whether the predetermined condition is satisfied according to;
[3] The first information is information about an image captured in a predetermined space, the second information is information about the temperature or carbon dioxide concentration in the predetermined space, and a specific event is in the predetermined space The detection system according to the above [1] or [2], which is a dozing state or a sign of dozing in a person;
[4] A third information obtaining means for obtaining third information different from the first information and the second information from the predetermined space, wherein the condition determining means obtains the obtained first information and the second information; and the third information satisfy a predetermined condition indicating that a specific event has occurred in a predetermined space, and the first information is an image of the predetermined space. The second information is information about the thermal image in the predetermined space, the third information is information about the carbon dioxide concentration in the predetermined space, and a specific event occurs in the predetermined space , The detection system according to the above [1] or [2], wherein there is an item left behind;
[5] The first information is information about an image obtained by capturing a predetermined space, the second information is information about a thermal image in the predetermined space, and a specific event is a person in the predetermined space. is present, the detection system according to the above [1] or [2];
[6] A third information obtaining means for obtaining third information different from the first information and the second information from the predetermined space, wherein the condition determining means obtains the obtained first information and the second information and the third information satisfy a predetermined condition indicating that a specific event has occurred in a predetermined space, and the first information is an image of the predetermined space. is information about
The second information is information about the thermal image in the predetermined space, the third information is information about the odor in the predetermined space, and the specific event is the presence of animals other than humans in the predetermined space. The detection system according to [1] or [2] above;
[7] The first information is information about an image captured in a predetermined space, the second information is information about a thermal image in the predetermined space, and a specific event causes heat generation in the predetermined space. The detection system according to [1] or [2] above, wherein a person is present;
[8] The first information is information about an image captured in a predetermined space, the second information is information about an odor in the predetermined space, and a specific event is when a person or animal is in the predetermined space. The detection system according to the above [1] or [2], which is vomiting;
[9] The above [1] to [8], including information about the sound generated in the predetermined space as information for determining whether or not a condition indicating that a specific event has occurred in the predetermined space is satisfied. ] The detection system according to any one of;
[10] Any one of the above [3] to [9], wherein the first condition determination means determines whether or not a predetermined first condition is satisfied based on the feature amount extracted from the captured image. detection system;
[11] The computer device is a server device capable of communication connection with another computer device, and the fact that a specific event has occurred when the server device is determined by the condition determination means to satisfy a predetermined condition. The detection system according to any one of the above [1] to [10], further comprising event information transmission means for transmitting the information of the above to another computer device. ;
[12] A detection method performed in a detection system comprising at least one computing device, comprising: a first information obtaining step of obtaining first information from a predetermined space; and a first information from the predetermined space. a second information acquisition step of acquiring different second information, and whether the acquired first information and second information satisfy a predetermined condition indicating that a specific event has occurred in a predetermined space A detection method, comprising a condition determination step of determining
[13] A space utilization system comprising at least one computer device and allowing a user to use a predetermined space, wherein among information obtainable from an information acquisition device installed in the predetermined space, the user Select the information to be actually acquired when using the specified space, or select the information to be calculated by the computer device from the information that can be acquired from the information acquisition device installed in the specified space. a space utilization system comprising selection means for
[14] The above-mentioned [13], comprising a specific granting means for granting a privilege to a user who uses a predetermined space, wherein the privilege granting means grants different types and/or different amounts of privilege according to the selected information. space utilization system according to;
[15] The space utilization system according to [14] above, wherein the privilege can be utilized in connection with the use of a predetermined space by the user;
[16] A space utilization method executed in a space utilization system that includes at least one computer device and allows a user to utilize a predetermined space, wherein When the user uses the specified space, the user selects the information that is actually acquired from the information, or the information that can be acquired from the information acquisition device installed in the specified space and is calculated by the computer device. A method of utilizing space, comprising a selection step of selecting information targeted for

ADVANTAGE OF THE INVENTION According to this invention, the detection system and detection method which can detect with high precision that the specific event generate|occur|produced in the predetermined space can be provided. ADVANTAGE OF THE INVENTION According to this invention, the novel space utilization system and space utilization method which give a privilege to the user who utilizes predetermined space can be provided.

1 is a block diagram showing the configuration of a detection system according to an embodiment of the invention; FIG. 1 is a block diagram showing the configuration of a server device according to an embodiment of the present invention; FIG. It is a figure which shows the flowchart of the 1st detection process concerning embodiment of this invention. It is a figure which shows the flowchart of the 2nd detection process concerning embodiment of this invention. It is a figure which shows the flowchart of the privilege provision process concerning embodiment of this invention.

Embodiments of the present invention will be described below, but the present invention is not limited to the following embodiments as long as they do not violate the gist of the present invention. The detection system of the present invention can detect that a specific event has occurred in a predetermined space, but the predetermined space is not particularly limited. In the detection system and space utilization system of the present invention, the predetermined space may be, for example, a moving body, the surroundings of the moving body, or the inside of the moving body. The following description mainly assumes that the detection system and space utilization system of the present invention are used for car sharing and car rental services.

As a moving body, it is used not only in vehicles such as automobiles, but also in aircraft such as drones, helicopters, airships, gliders, and jets, ships such as ferries and cargo ships, and public transportation such as buses and trains. Vehicles in general, such as vehicles, in which people can board can be mentioned. In other words, the moving object is a concept that includes not only vehicles but also the aircraft and ships described above.

FIG. 1 is a block diagram showing configurations of a detection system and a space utilization system according to an embodiment of the present invention. As shown in the figure, the detection system includes a server device 1, an information acquisition device 3 provided in a vehicle 2, and a communication network 6. FIG. The server device 1 can communicate with other computer devices, that is, the user terminal 4 or the administrator terminal 5 via the communication network 6 . The server device 1 will be described later.

The vehicle 2 is not particularly limited as long as it can move on land, but preferably has wheels. Examples of the vehicle 2 include not only bicycles, motorized bicycles, motorcycles, and automobiles, but also construction vehicles such as dump trucks and bulldozers, industrial vehicles such as forklifts used for cargo handling and transportation, and agricultural vehicles such as tractors. The concept also includes those used for special purposes such as agricultural vehicles used for

In one vehicle 2, the information acquisition device 3 includes at least a first information acquisition device 3a and a second information acquisition device 3b. Three or more information acquisition devices may be provided, such as 3d and a fifth information acquisition device 3e. The information acquisition device 3 may include a plurality of devices of the same type (that is, devices capable of acquiring the same information), or may include devices of a plurality of different types.

The information acquisition device 3 is not particularly limited as long as information around the information acquisition device 3 can be acquired. The information acquisition device 3 makes it possible to acquire information about the vehicle 2 , the surroundings of the vehicle 2 , or the inside of the vehicle 2 (inside the vehicle). Here, the information about the vehicle 2 is information about the whole or a part of the vehicle 2, and is a concept including information about parts and devices constituting the vehicle such as the tires and the engine of the vehicle 2, or the vicinity thereof. Information about the surroundings of the vehicle 2 is information about the outside of the vehicle 2 .

The space utilization system of the present invention acquires first information and second information different from the first information from the vehicle 2, the surroundings of the vehicle 2, or the inside of the vehicle 2 by the information acquisition device 3, and the second information satisfy a predetermined condition indicating that a specific event has occurred. More specifically, depending on whether the first information satisfies a predetermined first condition and whether the second information satisfies a predetermined second condition, whether a specific event has occurred It may be determined whether The first information and the second information may be obtained from the same information obtaining device 3, or the first information and the second information may be obtained from different information obtaining devices 3. FIG.

Further, the space utilization system of the present invention acquires the first information, the second information, and the third from the vehicle 2, the surroundings of the vehicle 2, or the inside of the vehicle 2 by the information acquisition device 3, and acquires the first information , the second information, and the third information satisfy a predetermined condition indicating that a specific event has occurred. More specifically, whether the first information satisfies a predetermined first condition, whether the second information satisfies a predetermined second condition, and whether the third information satisfies a predetermined third condition It may be determined whether or not a particular event has occurred depending on whether or not is satisfied. As described above, the first information, the second information, and the third information may be acquired from the same information acquisition device 3, and the first information, the third information, and the like may be acquired from different information acquisition devices 3. 2 information and 3rd information may be acquired.

In the space utilization system of the present invention, the number of pieces of information to be determined as to whether or not a predetermined condition indicating that a specific event has occurred is satisfied is not particularly limited as long as it is two or more. Whether or not a specific event has occurred may be determined according to whether or not each of the plurality of information acquired from the information acquisition device 3 satisfies a condition set for each piece of information.

Examples of the information acquisition device 3 include a driver monitoring system (DMS) and a vehicle-mounted device. With DMS, it is possible to detect whether the driver is in an appropriate state. The vehicle-mounted device can acquire position information by GPS or the like and acceleration information by an acceleration sensor.

In addition, the information acquisition device 3 may be, for example, a gas sensor, an alcohol checker, a body temperature sensor, a temperature sensor, or the like. The gas sensor can detect air pollution, the amount of carbon dioxide in the vehicle 2, and the like. Also, the alcohol checker can detect the concentration of alcohol contained in the user's breath. The temperature sensor can measure the temperature inside the vehicle 2 .

Further, as the information acquisition device 3, there is an imaging device capable of imaging a still image or a moving image. From the difference in image information before use, during use, or after use of the inside of the vehicle 2, or by using AI etc., it is possible to determine whether there is dirt, whether there is something left behind, whether there is an act such as vomiting, whether an infant etc. is on board. It is possible to detect whether or not the data is left unattended. It is possible to detect that a person, an animal, or the like is moving from changes over time in image information acquired by an imaging device.

The mounting position of the imaging device is not particularly limited. The imaging device can be placed in the center of the ceiling to capture all the states of the driver's seat, passenger's seat, and rear seats. A plurality of imaging devices may be arranged in each. Examples of imaging devices include an infrared camera and a 360-degree camera.

By the way, if the user hides the imaging device with tape or the like, the state inside the vehicle 2 cannot be captured accurately. For example, in the imaging device, it is possible to detect that the imaging device is hidden by using the fact that the illuminance detection becomes impossible as a trigger. When it is detected that the imaging device is hidden, it is possible to alert the user with an alarm by sound output or illumination.

For example, as will be described later, in a car sharing service or a rental car service, even when points are given to the user according to the use of the vehicle 2, as a result of the user putting a tape on the imaging device, When it is detected that the imaging device has been hidden for a certain period of time or more, measures such as reducing points given to the user or not giving points can be taken.

Further, as the information acquisition device 3, a tire sensor installed in a tire (or in the vicinity of the tire) of the vehicle 2 can measure and acquire the air pressure and temperature inside the tire. It is also possible to estimate the presence or absence of a puncture based on the information acquired from the tire sensor. Further, as the information acquisition device 3, an engine sensor installed in the engine of the vehicle 2 (or in the vicinity of the engine) can measure and acquire the rotation speed and temperature of the engine.

In addition to these, the information acquisition device 3 may be a drive recorder. Video and audio from the outside of the vehicle (in front or behind the vehicle) for several tens of seconds before and after the vehicle is subjected to a large impact, such as when the vehicle collides with an obstacle and stops suddenly, using a drive recorder. can be obtained.

As the information acquisition device 3, not only the voice sensor provided in the drive recorder described above but also a voice sensor installed separately from the drive recorder can be used. The voice sensor detects voice collected from a microphone provided inside the vehicle 2 . A sound sensor can detect and analyze sounds in the vehicle 2 . To improve the accuracy of determining whether an object present in the vehicle 2 is a person or an animal by voice analysis, or to determine whether an object present in the vehicle 2 is a forgotten item or a person. Judgment accuracy can be improved. In addition, speech analysis enables speech recognition that converts utterances into text.

In addition, the presence or absence of danger or violence in the vehicle 2 can be estimated by combining image information captured by the imaging device and voice analysis. Furthermore, by combining position information from a position sensor such as GPS, acceleration information from an acceleration sensor, and voice analysis, accidents and near misses can be detected.

The vehicle 2 is equipped with all of the DMS, GPS, acceleration sensor, gas sensor, alcohol checker, body temperature sensor, temperature sensor, imaging device, tire sensor, drive recorder, and voice sensor exemplified here as the information acquisition device 3. may be provided, or only a part thereof may be provided. Further, the information acquisition device 3 may include sensors and measurement devices other than those exemplified here.

Each information acquisition device 3 has a wireless communication function, and the information acquired by the information acquisition device 3 may be transmitted to the server device 1 by this communication function. In addition, each of the information acquisition devices 3 does not have a long-distance communication function, but the communication device provided in the vehicle 2 and the information acquisition device 3 perform short-distance communication by wireless or wire, and the communication device is used. It is also possible to transmit the information acquired by each of the information acquisition devices 3 to the server device 1 via the information acquisition device 3 .

In some cases, the information acquired by the information acquisition device 3 is processed by an arithmetic device provided in the vehicle 2, and only the information of the feature amount is transmitted to the server device 1 side. Further, based on the feature amount information processed by the arithmetic unit, a threshold value may be further judged, and only information such as the pass/fail result may be transmitted to the server device 1 side. As a result, information specifying the user is not stored in the server device 1, so the user can use the vehicle without worrying about privacy. The information acquired by the information acquisition device 3 or the information processed by the arithmetic device based on this information may be encrypted from the viewpoint of security before being transmitted to the server device 1 side. good.

It should be noted that it is also possible to rewrite the thresholds necessary for various determinations in the computing device via the cloud or using remote communication means.

Acquisition of information by the information acquisition device 3 can also be performed periodically at predetermined time intervals. Further, the acquisition of information by the information acquisition device 3 may be triggered by a predetermined operation of the vehicle 2 by the user (for example, unlocking the door, starting the engine, etc.). The frequency and timing of information acquisition by the information acquisition device 3 may be the same for all information acquisition devices 3, and the frequency and timing may vary depending on the type of information acquisition device 3. It may be done with

When the vehicle 2 is used for car sharing or car rental services, the reservation period reserved by the user of the vehicle 2, a certain period immediately before the reservation period, and further immediately after the reservation period. It is also possible to set the information acquisition device 3 to automatically acquire information only during a certain period of time.

The detection system and space utilization system of the present invention are provided with at least two information acquisition devices 3, and the combination thereof can also be appropriately designed according to the event to be detected.

For example, in a state in which the engine is running, the combination of the image information of the driver acquired by the DMS and the carbon dioxide concentration and/or the temperature inside the vehicle 2 acquired by the gas sensor indicates that the driver is dozing off. It is possible to detect the state and signs of dozing off. More specifically, based on the information of the driver's face image acquired by the DMS, it detects the occurrence of yawning or the continuation of the state where the eyes are closed, and also detects the inside of the vehicle 2 acquired by the gas sensor. When the concentration of carbon dioxide in the vehicle 2 is equal to or higher than a threshold value and/or when the temperature inside the vehicle 2 is equal to or higher than a predetermined temperature, it is possible to detect the driver's dozing state or a sign of dozing. Whether or not the engine is operating can be detected by an engine sensor (the same applies hereinafter).

Also, in a state where the engine is not running, it is possible to detect the presence or absence of a forgotten item by combining the image information obtained by the imaging device inside the vehicle 2 with the image information related to the thermal image inside the vehicle 2 acquired by the thermo camera. . More specifically, the image information captured in a state in which the engine of the vehicle 2 is not operating is different from the image information in the initial state before the start of use of the vehicle 2, and does not exist in the initial state. Detects the presence of an object. Furthermore, it is detected that the captured image information does not change with the passage of time, and that the object is in a stationary state without movement. If the concentration of carbon dioxide in the vehicle 2 does not change over time and the body temperature of the creature is not detected from the thermal image of the thermosensor, it can be detected that an item has been left behind. The analysis of the sound acquired by the sound sensor is used when the presence of an object that did not exist in the initial state is detected from the image information captured while the engine of the vehicle 2 is not operating. Thus, it is possible to improve the accuracy of determining whether an object remaining in the vehicle 2 is a person, an animal, or a forgotten item.

Also, in a state where the engine is not running, the image information inside the vehicle 2 acquired by the imaging device and the image information related to the thermal image inside the vehicle 2 acquired by the thermo camera are combined to prevent a child such as an infant from being left behind. can be detected. More specifically, when it is detected that the image information in the vehicle 2 acquired by the imaging device changes over time, and the body temperature of the creature is detected by the thermal image of the thermosensor, a child such as an infant is detected. can detect the possibility of being left behind.

Further, by combining image information inside the vehicle 2 acquired by an imaging device, odor inside the vehicle 2 acquired by a gas sensor, and image information related to a thermal image inside the vehicle 2 acquired by a thermo camera, animals such as pets can be detected. Boarding can be detected. More specifically, the image information inside the vehicle 2 acquired by the imaging device changes over time, and it is detected that the object is in motion, and the strength of the odor acquired by the gas sensor is detected. is equal to or greater than the threshold (or larger than the threshold), and furthermore, when the body temperature of the creature is detected from the thermal image of the thermosensor, it is possible to detect that an animal other than a human being, such as a pet, is on board. At this time, whether the creature remaining in the vehicle 2 is a human or an animal such as a pet can be determined with improved accuracy by utilizing the analysis of the voice acquired by the voice sensor.

Furthermore, by combining image information inside the vehicle 2 acquired by the imaging device and thermal image information related to the thermal image inside the vehicle 2 acquired by the thermo camera, it is possible to detect that a feverish person is in the vehicle. More specifically, when the presence of movement is detected from the image information in the vehicle 2 acquired by the imaging device, and the body temperature detected by the thermal image of the thermosensor is equal to or higher than a predetermined body temperature, It is possible to detect the possibility that a feverish person is on board. In addition, from the image information acquired by the imaging device, if AI or the like can detect the action of coughing or sneezing, or if the generation of droplets accompanying coughing or sneezing can be detected, a person who is in poor physical condition can get on the vehicle 2. You can detect what is happening.

Further, it is possible to detect vomiting of a person or an animal by combining image information inside the vehicle 2 acquired by the imaging device and information on the odor inside the vehicle 2 acquired by the gas sensor. More specifically, it is detected that the image information in the vehicle 2 acquired by the imaging device is different from the image information in the initial state before the user of the vehicle 2 starts using the vehicle 2 in a state where no human or animal is on board. , the occurrence of vomiting or other fouling can be detected when the strength of the odor obtained by the gas sensor is equal to or greater than the threshold (or greater than the threshold). In addition, when voice recognition detects utterances such as "I threw up", "I got dirty", "I spilled" in the vehicle 2, it detects with high accuracy that cleaning is necessary. be able to.

In addition, the judgment criteria and data thresholds for judging whether or not these specific events have occurred are based on numerous demonstration experiments and operational results, and after reflecting the voices of users. An administrator (or an administrator of a car sharing or car rental service) who operates and manages the detection system and the space utilization system can appropriately determine them. In addition, AI (artificial intelligence) is made to perform machine learning using these demonstration experiments, operation results, user comments, etc. as teaching data, and AI determines whether or not predetermined conditions are met, and specifies appropriate threshold values. can also

FIG. 2 is a block diagram showing the configuration of the server device according to the embodiment of the present invention. The server device 1 includes at least a control unit 11, a RAM 12, a storage unit 13, and a communication interface 14, which are connected by an internal bus.

The control unit 11 includes a CPU and a ROM, executes programs stored in the storage unit 13 , and controls the server device 1 . Further, the control unit 11 has an internal timer for measuring time. A RAM 12 is a work area for the control unit 11 . The storage unit 13 is a storage area for storing programs and data. The control unit 11 reads programs and data from the RAM 12 and executes program execution processing based on information received from the information acquisition device 3 and the like.

The communication interface 14 can be connected to the communication network 3 wirelessly or by wire, and can transmit and receive data to and from the server device 1 via the communication network 6 . Data received via the communication interface 14 is loaded into the RAM 12 and processed by the control unit 11 .

The user terminal 4 is a terminal operated by a user using the vehicle 2 . The administrator terminal 5 is a terminal operated by an administrator who operates and manages the detection system and/or the space utilization system (or an administrator who operates and manages car sharing and car rental services). When a specific event occurs in the vehicle 2, the server device 1 notifies the user terminal 4 and/or the administrator terminal 5 of information indicating that the specific event has occurred. The user terminal 4 and/or the administrator terminal 5 is not particularly limited as long as it is a computer device having a display screen and an input unit. mentioned. The user terminal 4 and/or the administrator terminal 5 can be connected to the communication network 6 wirelessly or by wire, and can transmit/receive data to/from the server device 1 via the communication network 6 .

FIG. 3 is a diagram showing a flowchart of first detection processing according to the embodiment of the present invention. The first detection process is executed by the server device 1 or an arithmetic device provided in the vehicle 2 . Based on the information acquired by the information acquisition device 3, the server device 1 or the arithmetic device executes the following processes. It should be noted that the following processing is periodically executed at predetermined time intervals.

First, based on the face image data of the driver acquired by the DMS, it is determined whether or not yawning occurs within a predetermined period of time and whether or not the eyes are kept closed (step S1). The determination of whether the driver yawns or continues to keep his eyes closed is performed using AI learned by machine learning such as deep learning. When it is determined that neither yawning nor continuation of the closed-eye state has occurred (No in step S1), the fact that no problem has occurred in the vehicle 2 is notified in the server device 1 or the arithmetic device. It is registered as history information (step S2), and the first detection process ends. The history information is registered by associating the user using the vehicle 2 with an identifiable user ID and/or associating the vehicle 2 with an identifiable vehicle ID. The same applies to the following history information registration.

On the other hand, if it is determined that the yawning and closed-eye state continues (Yes in step S1), whether the carbon dioxide concentration in the vehicle 2 obtained by the gas sensor is equal to or greater than the threshold value (or greater than a threshold) is determined (step S3). It should be noted that in step S3, instead of the carbon dioxide concentration, it is also possible to determine whether the temperature inside the vehicle 2 obtained by the temperature sensor is equal to or higher than a threshold value (or whether it is greater than the threshold value). Alternatively, it may be determined whether both conditions of whether the carbon dioxide concentration in the vehicle 2 is equal to or higher than the threshold and whether the temperature in the vehicle 2 is equal to or higher than the threshold are satisfied.

If the determination in step S3 is No, the fact that no problem has occurred in the vehicle 2 is registered as history information in the server device 1 or the arithmetic device (step S4), and the first detection process ends.

On the other hand, if it is determined as Yes in step S3, the driver is dozing off or there is a sign of dozing off, so that information is sent to the user terminal 4 and/or the administrator terminal 5 (step S5). . Then, in the vehicle 2, the fact that the driver has fallen asleep or has a sign of falling asleep is registered as history information in the server device 1 or the arithmetic device (step S6), and the first detection process ends.

FIG. 4 is a diagram showing a flowchart of second detection processing according to the embodiment of the present invention. The second detection process is executed by the server device 1 or an arithmetic device provided in the vehicle 2 . Based on the information acquired by the information acquisition device 3, the server device 1 or the arithmetic device executes the following processes. It should be noted that the following processing is periodically executed at predetermined time intervals while the vehicle 2 is being used by the user.

First, it is determined whether or not the engine is operating based on information such as the number of revolutions of the engine acquired by the engine sensor (step S11). If the engine is not in operation (No in step S11), it is determined whether or not the image information captured by the imaging device is different from the image information in the initial state before the start of use of the vehicle 2 (step S12). Here, the image information in the initial state refers to image information captured before the user starts using the vehicle 2 and in a state where no one is in the vehicle 2 . For example, when a signal requesting unlocking of the door of the vehicle 2 is received from the outside as a trigger, the interior of the vehicle 2 is imaged immediately after receiving the signal, thereby acquiring image information of the initial state. can be done.

For example, a background subtraction method can be used to determine whether the image information captured by the imaging device in step S12 is different from the image information in the initial state before the vehicle 2 is started to be used. For example, there is a method of obtaining a difference image by using image information in an initial state as a background image and calculating the absolute value of the difference from an image captured by an imaging device. When a difference image is obtained, it can be seen that the image captured by the imaging device is different from the image in the initial state. In addition to the background subtraction method, an edge detection method can also be used. By extracting the contours of both the image in the initial state and the image captured by the imaging device by the edge detection method, and comparing the contours extracted from each image, the image information captured by the imaging device is It is possible to determine whether or not the image information is different from the image information in the initial state before the start of use of the vehicle 2 .

If it is determined that the image information captured by the imaging device does not differ from, or is the same as, the image information in the initial state before the start of use of the vehicle 2 (No in step S12), the odor acquired by the gas sensor is detected. It is determined whether or not the strength is equal to or greater than the threshold (step S13). If the intensity of the odor obtained by the gas sensor is less than the threshold value (No in step S13), the server device 1 or the arithmetic device indicates that no specific event has occurred and that no problem has occurred in the vehicle 2. is registered as history information (step S14), and the second detection process ends.

If the strength of the odor obtained by the gas sensor is equal to or greater than the threshold value (Yes in step S13), there is a possibility that dirt due to vomiting or the like has occurred, and the user terminal notifies that there is dirt due to vomiting or the like inside the vehicle 2. 4 and/or to the administrator terminal 5 (step S15). Then, in the vehicle 2, the occurrence of dirt due to vomiting or the like is registered as history information in the server device 1 or the arithmetic device (step S16), and the second detection process ends.

When it is determined that the image information captured by the image capturing device is different from the image information in the initial state before the vehicle 2 is started to be used (Yes in step S12), the image information captured by the image capturing device does not match the image information of the time. It is determined whether or not there is a change over time (step S17). The determination in step S17 can also use the background subtraction method or the edge detection method as in step S12.

If it is determined that the image information captured by the imaging device has not changed over time (No in step S17), the concentration of carbon dioxide in the vehicle 2 measured by the gas sensor has changed over time. It is determined whether or not there is (step S18). If the concentration of carbon dioxide in the vehicle 2 has not changed over time (No in step S18), it is determined whether or not there is data corresponding to a temperature equal to or higher than a predetermined temperature in the thermal image data acquired from the thermo camera. (step S19). Here, the predetermined temperature can be set as appropriate, and can be set to 35° C., for example. The body temperature of warm-blooded animals, including humans, is approximately 35 to 43° C. By setting a predetermined temperature in this way, it is possible to detect the presence of many warm-blooded animals, including humans.

In the thermal image data acquired from the thermo camera, if there is no data corresponding to a temperature equal to or higher than the predetermined temperature (No in step S19), there is a high possibility that the user left behind something in the vehicle 2. A message indicating that the user left behind an item is transmitted to the user terminal 4 and/or the administrator terminal 5 (step S20). Then, in the vehicle 2, the fact that the driver or fellow passenger left behind an item is registered as history information in the server device 1 or the arithmetic device (step S21), and the second detection process ends.

In the thermal image data acquired from the thermo camera, if there is data corresponding to a temperature equal to or higher than the predetermined temperature (Yes in step S19), it is unlikely that a child such as an infant is left behind in the vehicle 2, but some Since there is a possibility that a trouble has occurred, a message to that effect is transmitted to the user terminal 4 and/or the administrator terminal 5 (step S22). Then, the fact that there is a possibility of trouble occurring in the vehicle 2 is registered as history information in the server device 1 or the arithmetic device (step S23), and the second detection process ends.

If it is determined that the image information captured by the imaging device has changed over time (Yes in step S17), or if the concentration of carbon dioxide in the vehicle 2 changes over time (step S18 Yes), there is a possibility that a child such as an infant or a pet has been left behind in the vehicle 2, and a message to that effect is transmitted to the user terminal 4 and/or the administrator terminal 5 that the child or pet may be left behind. (Step S24). Then, in the vehicle 2, the fact that the child may be left behind is registered as history information in the server device 1 or the arithmetic device (step S25), and the second detection process ends. In addition, in step S24, before transmitting to the user terminal 4 and/or the administrator terminal 5 that there is a possibility that a child or pet has been left behind, the strength of the odor obtained by the gas sensor is used to It may be determined whether the object is a child or a pet.

Next, the case where it is determined as Yes in step S11 will be described. It is determined whether or not the image information captured by the imaging device changes over time (step S26). The determination in step S26 can also use the background subtraction method or the edge detection method, as in step S13. If it is determined that the image information captured by the imaging device has not changed over time (No in step S26), the server device 1 or the computing device indicates that no problem has occurred in the vehicle 2. It is registered as history information (step S27), and the second detection process ends.

If it is determined that the image information captured by the imaging device changes over time (Yes in step S26), the thermal image data obtained from the thermo camera does not include data corresponding to a temperature equal to or higher than a predetermined temperature. It is determined whether or not it exists (step S28). In step S28, 37° C. can be set as the predetermined temperature. The body temperature of many warm-blooded animals other than humans is 37° C. or higher, and it is possible to determine whether or not there is a possibility that a febrile person or animal is present in the vehicle 2 in step S28.

In the thermal image data, if there is no data corresponding to a temperature equal to or higher than a predetermined temperature (No in step S28), it is not possible to confirm the presence of a person with fever or an animal in the vehicle 2, so that no particular problem has occurred. The history information is registered in the server device 1 or the arithmetic device (step S29), and the second detection process ends.

On the other hand, if there is data corresponding to a temperature equal to or higher than the predetermined temperature in the thermal image data (Yes in step S28), it is determined whether the intensity of the odor obtained by the gas sensor is equal to or greater than the threshold (or greater than the threshold). is determined (step S30). If the strength of the odor obtained by the gas sensor is less than the threshold value (No in step S30), the presence of a feverish person in the vehicle 2 is transmitted to the user terminal 4 and/or the administrator terminal 5 (step S31). Then, in the vehicle 2, the presence of a feverish person is registered as history information in the server device 1 or the arithmetic device (step S32), and the second detection process ends.

If the strength of the odor obtained by the gas sensor is equal to or greater than the threshold value (Yes in step S30), the fact that the pet is in the vehicle 2 is transmitted to the user terminal 4 and/or the administrator terminal 5 (step S33). ). Then, in the vehicle 2, the fact that the pet is on board is registered as history information in the server device 1 or the arithmetic device (step S34), and the second detection process ends.

The predetermined conditions in the detection system of the present invention may be determined by the administrator of the detection system and registered in the server device 1. By operating , and transmitting the input conditions to the server device 1 , it is possible to register the conditions defined by the user in the server device 1 .

Next, the space utilization system of the present invention will be explained. In a car sharing service or rental car service that introduces the detection system and space utilization system of the present invention, a user can reserve the use of the vehicle 2 . When making a reservation for use of the vehicle 2 , it is possible to select from which information acquisition device 3 of the plurality of information acquisition devices 3 provided in the vehicle 2 the acquisition of information is permitted. Alternatively, when making a reservation for use of the vehicle 2, any of the plurality of information acquired from the information acquisition device 3 provided in the vehicle 2 is selected as an analysis target in the server device 1 (for example, the above-described first It is possible to select whether the detection processing or the calculation target in the second detection processing).

Many users have psychological resistance to acquiring and utilizing information on the vehicle 2 . Therefore, in order to give the user an incentive to utilize the information of the vehicle 2, a large number of points can be given to the user who is permitted to obtain or use the information from the information obtaining device 3. In addition, among the information acquisition devices 3, for example, there are devices such as an imaging device that tend to cause greater psychological resistance when used. Therefore, by changing the number of points to be given to the user according to the type of the information acquisition device 3, or by giving points with different rarity and value to the user, it is possible to obtain or use information by the information acquisition device 3. can encourage

Next, the space utilization system of the present invention will be explained. FIG. 5 is a diagram showing a flowchart of reservation processing in the space utilization system according to the embodiment of the present invention. A user who wishes to use a car sharing service or a rental car operates the user terminal 4 and accesses a predetermined website. A user ID and password are registered for each user, and the user logs in to the website by entering the user ID and password into the user terminal 4 (step S41).

When the user logs in to the website and inputs conditions such as usage date and time and budget, information on a plurality of vehicles that can be reserved that match the conditions is transmitted from the web server and displayed on the user terminal 4 (step S42). ). The vehicle information includes, for example, vehicle type, number of passengers, cost, external image of the vehicle, and the like. By operating the user terminal 4, the user can select one vehicle for reservation from a plurality of vehicles (step S43). Next, it is possible to select which information acquisition device from among the plurality of information acquisition devices provided in the vehicle is permitted to acquire or use information when actually using the vehicle (step S44). . Note that "selection of information" is a concept that includes not only selecting information such as image information captured by an imaging device and audio information acquired by an audio sensor, but also selecting an information acquisition device such as an imaging device and an audio device. is.

When the selection is completed in step S44, the points given to the user are calculated (step S45). The given points can be used to reduce the cost for the user to use the car sharing service or the rental car service from the next time onward, or to make a reservation with priority over other users. Points may award different types and/or different amounts of benefits depending on the information selected. The point calculation method can be set as appropriate. For example, the greater the number of pieces of information permitted to be used, the more points can be given. For example, points are determined for each piece of information that can be selected, and points are added according to the selected information. More points can be given to information such as image information acquired by an imaging device that causes greater psychological resistance to use than other information. Also, different types of points can be given according to the information that is permitted to be used.

Next, information about the confirmed reservation is registered in the server device 2 (step S46). In the server device 2, the date and time of use of the vehicle, the reserved vehicle, and the information permitted to be obtained are recorded in association with the user ID. Further, in the server device 2, the points calculated in step S45 are added in association with the user ID. The reservation processing is completed by the processing of steps S41 to S46.

In step S44, the user can operate the user terminal 4 to select from which information acquisition device 3 acquisition or use of information is permitted. In addition, information to be actually acquired or used by the information acquisition device 3 can be selected. For example, the server device 1 can automatically select whether to permit the acquisition or use of information from the information acquisition device 3 according to user attributes and information registered in advance by the user.

In the embodiment described above, when the user makes a reservation to use the vehicle 2, the user selects from which information acquisition device 3 the acquisition or use of information is permitted. It may be possible to select from which information acquisition device 3 the acquisition or use of information is permitted at a time different from the time of making a reservation for use. For example, immediately before, immediately after, or during use of the vehicle 2, it may be possible to select from which information acquisition device 3 acquisition or use of information is permitted.

In the embodiments described above, application of the detection system and space utilization system of the present invention was mainly applied to car sharing services and rental car services, but they can also be preferably used in car leasing services. . Car leasing differs from car sharing in that the same user uses the same vehicle for a long period of time, but by using this detection system, the leasing company can grasp the deterioration status of the vehicle body. . The leasing company can use the information obtained from the detection system to evaluate the vehicle, and when the user returns the vehicle, it will be possible to present appropriate resale conditions to the user.

Also, the detection system and space utilization system of the present invention can detect that a specific event is occurring in a space within a moving object such as a vehicle. For example, it can be used in any space such as general homes, schools, hospitals, offices, factories, various shops, and public transportation.

According to the present invention, at least one computer device is provided, and it is determined whether the first information and the second information acquired from a predetermined space satisfy a predetermined condition indicating that a specific event has occurred. Therefore, it is possible to determine with higher accuracy than determining whether a specific event has occurred based on information obtained from one sensor or the like.

According to the present invention, the predetermined predetermined condition is satisfied depending on whether the obtained first information satisfies the predetermined first condition and whether the obtained second information satisfies the predetermined second condition. Since it is determined whether or not the condition is satisfied, it can be determined with higher accuracy whether or not a predetermined event has occurred in a predetermined space.

According to the present invention, the first information is information about an image obtained by capturing a predetermined space, and the second information is information about the temperature or carbon dioxide concentration in the predetermined space. It is possible to detect a dozing state of a person in the room or a sign of dozing.

According to the present invention, the first information is information about an image captured in a predetermined space, the second information is information about a thermal image in the predetermined space, and the third information is information about the predetermined space. By using the information about the carbon dioxide concentration in the space, it is possible to detect the presence of a forgotten item in the predetermined space.

According to the present invention, the first information is information about an image captured in a predetermined space, and the second information is information about a thermal image in the predetermined space. can be detected to exist.

According to the present invention, the first information is information about an image captured in a predetermined space, the second information is information about a thermal image in the predetermined space, and the third information is information about the predetermined space. The presence of an animal other than a human being in a predetermined space can be detected by using information related to the odor in the space.

According to the present invention, the first information is information about an image captured in a predetermined space, and the second information is information about a thermal image in the predetermined space. It is possible to detect that a person exists.

According to the present invention, the first information is information about an image captured in a predetermined space, and the second information is information about an odor in the predetermined space. Vomiting can be detected.

According to the present invention, the information about the sound generated in the predetermined space is included as the information for determining whether or not the condition indicating that the specific event has occurred in the predetermined space is satisfied. Whether an event has occurred can be determined with higher accuracy.

According to the present invention, whether or not the first information satisfies the predetermined first condition is determined based on the feature amount extracted from the captured image, so that the predetermined event occurs in the predetermined space. Whether or not it has occurred can be determined with higher accuracy.

According to the present invention, a user terminal operated by a user who uses a predetermined space and a server device capable of communication connection with the user terminal are provided, and when it is determined that a predetermined condition is satisfied, a specific event occurs. Since the occurrence is sent to the user terminal, the user can be notified that a particular event has occurred. Further, according to the present invention, an administrator terminal that operates and manages the detection system, and a server device that can be communicatively connected to the administrator terminal are provided. Since the fact that an event has occurred is sent to the administrator terminal, it is possible to notify the administrator/operator that a specific event has occurred.

According to the present invention, when a user uses a predetermined space from information that can be obtained from an information obtaining device installed in a predetermined space, the user selects information to be actually obtained, or Information to be processed by the computer can be selected from the information that can be acquired from the information acquisition device installed in the . Therefore, depending on the user who uses the predetermined space, or even if the same user uses the predetermined space, which information acquisition device is used to acquire information, or whether information is acquired or not. It is possible to select which information acquired from the device is to be used.

According to the present invention, when granting a privilege to a user who uses a predetermined space, by granting different types and/or different amounts of privilege according to the selected information, the information in the predetermined space Incentives can be given to the users for most of the information to be used by the administrator, or for information that the user does not actively desire to be used by the administrator.

According to the present invention, since the privilege can be utilized in relation to the use of the predetermined space by the user, it is possible to encourage the user to use the predetermined space.

1 server device, 2 vehicle, 3 information acquisition device, 4 user terminal,
5 administrator terminal, 6 communication network, 11 control unit, 12 RAM,
13 storage unit, 14 communication interface

Claims (16)

comprising at least one computing device;
a first information obtaining means for obtaining first information from a predetermined space;
a second information acquiring means for acquiring second information different from the first information from the predetermined space;
condition determination means for determining whether the acquired first information and second information satisfy a predetermined condition indicating that a specific event has occurred in a predetermined space;
the first information is information about an image obtained by capturing a predetermined space;
the second information is information about the temperature in the predetermined space;
A sensing system in which a particular event is a drowsy state or a precursor to drowsiness of a person within a given space. comprising at least one computing device;
a first information obtaining means for obtaining first information from a predetermined space;
a second information acquiring means for acquiring second information different from the first information from the predetermined space;
condition determination means for determining whether the acquired first information and second information satisfy a predetermined condition indicating that a specific event has occurred in a predetermined space;
the first information is information about an image obtained by capturing a predetermined space;
the second information is information about the carbon dioxide concentration in a given space;
A sensing system in which a particular event is a drowsy state or a precursor to drowsiness of a person within a given space. comprising at least one computing device;
a first information obtaining means for obtaining first information from a predetermined space;
a second information acquiring means for acquiring second information different from the first information from the predetermined space;
condition determination means for determining whether the acquired first information and second information satisfy a predetermined condition indicating that a specific event has occurred in a predetermined space;
the first information is information about an image obtained by capturing a predetermined space;
the second information is information about a thermal image in a predetermined space;
A sensing system in which the specified event is the presence of a person within a given space. comprising at least one computing device;
a first information obtaining means for obtaining first information from a predetermined space;
a second information acquiring means for acquiring second information different from the first information from the predetermined space;
condition determination means for determining whether the acquired first information and second information satisfy a predetermined condition indicating that a specific event has occurred in a predetermined space;
the first information is information about an image obtained by capturing a predetermined space;
the second information is information about a thermal image in a predetermined space;
A detection system wherein the specified event is the presence of a fever person within a given space. comprising at least one computing device;
a first information obtaining means for obtaining first information from a predetermined space;
a second information acquiring means for acquiring second information different from the first information from the predetermined space;
condition determination means for determining whether the acquired first information and second information satisfy a predetermined condition indicating that a specific event has occurred in a predetermined space;
the first information is information about an image obtained by capturing a predetermined space;
the second information is information about the odor in the predetermined space;
A detection system in which the specific event is vomiting by a person or animal, or the occurrence of soiling. comprising at least one computing device;
a first information obtaining means for obtaining first information from a predetermined space;
a second information acquiring means for acquiring second information different from the first information from the predetermined space;
a third information acquiring means for acquiring third information different from the first information and the second information from the predetermined space;
condition determination means for determining whether the acquired first information, second information, and third information satisfy a predetermined condition indicating that a specific event has occurred in a predetermined space;
the first information is information about a predetermined spatial position;
the second information is information about acceleration in a given space;
the third information is information about the sound generated in the predetermined space;
A detection system in which the specified event is an accident or near miss. The condition determination means is
a first condition determining means for determining whether the acquired first information satisfies a predetermined first condition;
a second condition determination means for determining whether the acquired second information satisfies a predetermined second condition;
Furthermore, the condition determination means
It is determined whether or not the predetermined condition is satisfied according to whether or not it is determined that the predetermined first condition is satisfied and whether or not the predetermined second condition is satisfied. 7. The detection system according to any one of 6. Any one of claims 1 to 5 and 7, wherein the information for judging whether or not a condition indicating that a specific event has occurred in the predetermined space is satisfied includes information on the sound generated in the predetermined space. The detection system described in . The first condition determination means is
9. The detection system according to any one of claims 1 to 5, 7 and 8, wherein it is determined whether or not a predetermined first condition is satisfied based on the feature amount extracted from the captured image. The computer device is a server device capable of communication connection with another computer device,
The server device
event information transmission means for transmitting information to the effect that a specific event has occurred to another computer device when the condition determination means determines that a predetermined condition is satisfied;
A sensing system according to any of claims 1-9. A detection method performed in a detection system comprising at least one computing device, comprising:
a first information acquisition step of acquiring first information from a predetermined space;
a second information acquisition step of acquiring second information different from the first information from the predetermined space;
a condition determination step of determining whether the acquired first information and second information satisfy a predetermined condition indicating that a specific event has occurred in a predetermined space;
the first information is information about an image obtained by capturing a predetermined space;
the second information is information about the temperature in the predetermined space;
A detection method, wherein a specific event is a dozing state or a sign of dozing of a person in a predetermined space. A detection method performed in a detection system comprising at least one computing device, comprising:
a first information acquisition step of acquiring first information from a predetermined space;
a second information acquisition step of acquiring second information different from the first information from the predetermined space;
a condition determination step of determining whether the acquired first information and second information satisfy a predetermined condition indicating that a specific event has occurred in a predetermined space;
the first information is information about an image obtained by capturing a predetermined space;
the second information is information about the carbon dioxide concentration in a given space;
A detection method, wherein a specific event is a dozing state or a sign of dozing of a person in a predetermined space. A detection method performed in a detection system comprising at least one computing device, comprising:
a first information acquisition step of acquiring first information from a predetermined space;
a second information acquisition step of acquiring second information different from the first information from the predetermined space;
a condition determination step of determining whether the acquired first information and second information satisfy a predetermined condition indicating that a specific event has occurred in a predetermined space;
the first information is information about an image obtained by capturing a predetermined space;
the second information is information about a thermal image in a predetermined space;
A detection method, wherein the specific event is the presence of a person within a predetermined space. A detection method performed in a detection system comprising at least one computing device, comprising:
a first information acquisition step of acquiring first information from a predetermined space;
a second information acquisition step of acquiring second information different from the first information from the predetermined space;
a condition determination step of determining whether the acquired first information and second information satisfy a predetermined condition indicating that a specific event has occurred in a predetermined space;
the first information is information about an image obtained by capturing a predetermined space;
the second information is information about a thermal image in a predetermined space;
A detection method, wherein the specific event is the presence of a febrile person within a given space. A detection method performed in a detection system comprising at least one computing device, comprising:
a first information acquisition step of acquiring first information from a predetermined space;
a second information acquisition step of acquiring second information different from the first information from the predetermined space;
a condition determination step of determining whether the acquired first information and second information satisfy a predetermined condition indicating that a specific event has occurred in a predetermined space;
the first information is information about an image obtained by capturing a predetermined space;
the second information is information about the odor in the predetermined space;
A method of detection wherein the specific event is vomiting by a person or animal, or the occurrence of soiling. A detection method performed in a detection system comprising at least one computing device, comprising:
a first information acquisition step of acquiring first information from a predetermined space;
a second information acquisition step of acquiring second information different from the first information from the predetermined space;
a third information acquisition step of acquiring third information different from the first information and the second information from the predetermined space;
a condition determination step for determining whether the acquired first information, second information, and third information satisfy a predetermined condition indicating that a specific event has occurred in a predetermined space;
the first information is information about a predetermined spatial position;
the second information is information about acceleration in a given space;
the third information is information about the sound generated in the predetermined space;
A detection method in which the specific event is an accident or a near miss.

Images