WO2020044797A1

WO2020044797A1 - Information processing device, information processing system, and information processing method

Info

Publication number: WO2020044797A1
Application number: PCT/JP2019/027077
Authority: WO
Inventors: 義浩細川; 文彦草野; 健一橋詰
Original assignee: 三菱電機株式会社
Priority date: 2018-08-27
Filing date: 2019-07-09
Publication date: 2020-03-05
Also published as: JPWO2020044797A1; JP6972362B2; CN112567305A; CN112567305B

Abstract

An information processing device is configured by: a data input unit to which identification information transmitted from a transmitter attached to a monitoring subject and radio waves received by a receiver are input as a reception radio wave intensity, and image coordinates and an image acquisition time of a camera image are input; a determination unit for searching for an estimation radio wave intensity based on the image coordinates from a reference database in which reference reception radio wave intensities for each operation are stored, and determining whether the reception radio wave intensities are within an acceptable range of the estimation radio wave intensity or not; a data recording unit for recording, in the case where the reception radio wave intensity is determined to be within the acceptable range, the reception radio wave intensity, the identification information, the image coordinates, and the image acquisition time; a data accumulation unit for accumulating the data of the data recording unit; and an analysis unit for analyzing a motion of the monitoring subject by using the reception radio wave intensity, the identification information, the image coordinates, and the image acquisition time accumulated in the data accumulation unit.

Description

Information processing apparatus, information processing system, and information processing method

The present disclosure relates to an information processing apparatus, an information processing system, and an information processing method for grasping a movement of a worker or the like using a transceiver.

情報処理 An information processing system has been developed in which receivers periodically receive radio waves transmitted from transmitters in order to grasp the movements of equipment, people, and things (products, tools, etc.).

In particular, in production management, a radio wave transmitted from an operator carrying a transmitter with identification information is received by a receiver installed in a work area, the operator is identified from the identification information, and the received radio wave intensity is determined. The system which grasps the contents of work from is used. For example, the distance from the transmitter to the receiver can be ascertained from the received radio wave intensity, and the movement history of the worker can be known.

For example, Patent Literature 1 discloses a technique in which a worker carries a sensor and automatically collects position information of the worker, thereby recording a time required for the work and a break in the work.

システム There is also a system that confirms the work time and work contents by checking the camera image in the work area. For example, there is a technique in which a worker inputs identification information and position information of the worker in advance and monitors the worker with a camera image. Further, there is disclosed a technique in which a camera recognizes a worker's face, inputs position information in advance, and grasps the contents of the work using a transmitter to which identification information is added (for example, see Patent Document 2). .

JP 2009-294732 A WO2017 / 013899

However, in the information processing system described above, since the radio wave transmitted from the transmitter is grasped by the distance of the worker based on the received radio wave intensity attenuated when the radio wave is received by the receiver, the position information of the worker is accurately obtained. It was difficult to grasp. Further, when there are a plurality of receivers, the receiver receives a plurality of radio waves transmitted from the transmitter, so that it may be difficult to grasp the exact position of the worker.

Also, although it is possible to identify the worker by recognizing the face from the image of the camera, a camera with a high resolution is required. In addition, even if a high-resolution camera is used, the worker cannot be identified when the worker faces downward or backward, and the face of the worker is registered in advance in order to recognize the face with an image. There was also a need.

The present disclosure has been made in order to solve such a problem, and an object of the present disclosure is to provide an information processing apparatus, an information processing system, and an information processing method that can accurately identify identification information of a monitoring target and movement of the monitoring target. And

The information processing apparatus according to the present disclosure is configured such that the identification information transmitted from the transmitter attached to the monitoring target and the radio wave received by the receiver are input as the received radio wave intensity, the image coordinates of the camera image, and the image acquisition time. Is searched for an estimated radio field intensity based on image coordinates from a data input unit to which is input and a reference database in which a reference received radio field intensity for each operation is stored, and whether the received radio field intensity is included in the allowable range of the estimated radio field intensity is determined. A data recording unit for recording received radio field intensity, identification information, image coordinates, and image acquisition time when it is determined that the received radio field intensity is included in the allowable range; and a data recording unit. Analysis that analyzes the movement of the monitoring target using the received signal strength, identification information, image coordinates, and image acquisition time stored in the data storage unit that stores the data of It is those with a door.

Further, an information processing system according to the present disclosure includes a transmitter attached to a monitoring target, a receiver for receiving identification information and radio waves transmitted from the transmitter, and an identification information transmitted from the transmitter attached to the monitoring target. And a data input unit into which the radio wave received by the receiver is input as the received radio wave intensity, the image coordinates of the image of the camera and the image acquisition time are input, and a reference database in which the reference received radio wave intensity for each work is stored. A determination unit that searches for an estimated radio field intensity based on the image coordinates from the image coordinates and determines whether the received radio field intensity is within an allowable range of the estimated radio field intensity, and determines that the received radio field intensity is included in the allowable range. In this case, a data recording unit for recording received radio field intensity, identification information, image coordinates, and an image acquisition time, a data storage unit for storing data of the data recording unit, and a data storage unit Received signal strength that is, those with the identification information, image coordinates, and an analysis unit for analyzing the motion of the monitored using an image acquisition time, and a display unit for displaying the results of the analysis by the analysis unit.

Further, the information processing method according to the present disclosure includes a step of obtaining a reference reception radio field intensity for each task together with a reference image coordinate to create a reference database, and a step of obtaining identification information and reception radio field intensity transmitted from a transmitter attached to a monitoring target. And the step of recording the image coordinates and the image acquisition time of the image acquired from the camera, and comparing the estimated radio wave intensity and the received radio wave intensity retrieved from the reference database based on the image coordinates, and A step of determining whether or not it is included in the allowable range, and a step of recording the received radio field intensity, identification information, image coordinates, and the image acquisition time as data if it is determined that it is included in the allowable range. And storing the recorded data as accumulated data, and analyzing the movement of the monitoring target using the accumulated data.

According to the present disclosure, the received radio field intensity received by the receiver is based on the image coordinates of the image of the camera, and falls within the allowable range of the estimated radio field intensity estimated from the reference database created as the reference data. By accumulating the image coordinates, image acquisition time, and identification information related to the received radio wave intensity, and using these accumulated data, the resolution of the camera, the state of the person carrying the transmitter or the state of the object accurately. The movement of the monitoring target can be grasped.

1 is a schematic configuration diagram illustrating an information processing system according to a first embodiment of the present disclosure. 1 is a schematic configuration diagram illustrating an information processing device according to a first embodiment of the present disclosure. 6 is a flowchart illustrating an outline of a processing procedure according to the first embodiment of the present disclosure. FIG. 9 is a schematic configuration diagram illustrating an information processing system according to a second embodiment of the present disclosure. 15 is a schematic diagram of a display example of the information processing system according to the third embodiment of the present disclosure. 13 is a flowchart illustrating an information processing step according to the fourth embodiment of the present disclosure. FIG. 13 is a diagram illustrating a relationship between a distance between a transmitter and a receiver and a received radio wave intensity according to a fourth embodiment of the present disclosure. 13 is a flowchart illustrating an information processing step according to the fourth embodiment of the present disclosure. 15 is a flowchart illustrating a process of creating a reference database according to the fourth embodiment of the present disclosure. 13 is a flowchart illustrating an information processing step according to the fourth embodiment of the present disclosure. 15 is a flowchart illustrating an information processing step according to the fifth embodiment of the present disclosure. 15 is a flowchart illustrating another information processing process according to the fifth embodiment of the present disclosure. 15 is a flowchart illustrating another information processing process according to the fifth embodiment of the present disclosure. 15 is a flowchart illustrating another information processing process according to the fifth embodiment of the present disclosure. 15 is a flowchart illustrating another information processing process according to the fifth embodiment of the present disclosure. It is the figure which looked at the production site in the state where the worker turned to the direction opposite to the receiver and the camera from vertically above. It is the figure which looked at the production site in the state where the worker turned to the direction of the receiver and the camera from the vertical upper part. 15 is a flowchart illustrating an outline of a processing procedure according to the sixth embodiment of the present disclosure. FIG. 2 is a diagram illustrating an example of a hardware configuration that executes some functions of an information processing device.

Embodiment 1 FIG.
An information processing system for monitoring a monitoring target such as a person or a thing according to the present disclosure will be described with reference to the drawings. FIG. 1 is a schematic configuration diagram illustrating an information processing system according to the first embodiment of the present disclosure, and illustrates an example of monitoring the work content of a worker using the information processing system. Depending on the situation, it is also possible to grasp not only the worker but also the product, the movement of the mobile trolley, or a part thereof.

(1) In the information processing system shown in FIG. 1, a camera 1 that overlooks a production site is installed, an image and an image acquisition time are acquired by the camera 1, and the image coordinates of the image become position information.

In the information processing system,

receivers

10 and 11 are installed, and these are radio waves transmitted from the transmitters 101 to 103 carried by the workers A, B, and C to be monitored, ie, the accompanying transmitters. Is received as the received signal strength. This received radio wave intensity is also the position information of the workers A to C. The transmitters 101 to 103 transmit radio waves and also transmit identification information allocated to each of them.

Then, one of a plurality of

monitoring frames

201, 202, and 203 in the image of the camera 1 is selected, and for example, the image coordinates of the monitoring target in the selected monitoring frame 201 are set as the position information. Then, the received radio wave intensity related to the position information is found, the position information is determined, and the worker A is determined from the identification information of the transmitter 101 that transmitted the radio wave.

FIG. 2 is a schematic configuration diagram showing an information processing apparatus, and FIG. 3 is a flowchart showing an outline of a processing procedure.

(4) The information processing device 3 is provided with a reference database 2220. For example, the worker R performs the work performed by the workers A to C one by one for reference, and the

receivers

10 and 11 receive the radio wave of the transmitter 106 carried by the worker R as the reference received radio wave intensity, Reference data that associates these with the reference image coordinates of the image acquired by the camera 1 is stored in the reference database 2220. The reference database 2220 records the reference image coordinates of the camera and the reference radio wave reception time concerning the work acquired in advance, and stores the reference reception radio wave intensity.

For example, when the worker A to be monitored starts work, the

receivers

10 and 11 receive the identification information and the radio wave transmitted from the transmitter 101 carried by the worker A as the received radio wave intensity (ST1001). Then, the radio waves received by

receivers

10 and 11 are recorded in radio field intensity database 2210 as received radio field intensity (ST1002). Since the radio wave is transmitted from the transmitter 101 carried by the worker A, the worker is determined to be A by the identification information received by the

receivers

10 and 11.

On the other hand, the worker A is displayed on the selected monitoring frame 201, acquires an image from the camera (ST1003), and records the image coordinates and the image acquisition time of the image in the image time database 2200 (ST1004).

As described above, since the reference database storing the reference reception radio field intensity for each work is created (ST1005), the reference reception radio field intensity stored in the reference database 2220 based on the image coordinates of the image of the worker A's camera is obtained. The estimated radio field intensity is searched (ST1006).

It is sequentially determined whether or not the received radio wave intensity of the worker A is within an allowable range of the estimated radio wave intensity. If the received radio wave intensity is within the allowable range, the image coordinates and the image acquisition time acquired by the camera 1 of the worker A are added, and monitoring is performed. The movement of the target worker A, that is, the work content, is recorded in the data group recording database 2230 (ST1007). The data group record database 2230 is rewritten each time as a buffer. This operation is repeated, and the data recorded in the data group recording database 2230 is accumulated in the data accumulation database 2240 as accumulated data. The work contents of the workers B and C can be similarly stored as data.

These operations are performed by the data input unit 31, the determination unit 32, the data recording unit 33, the data storage unit 34, the reference data creation unit 35, and the analysis unit 36 of the information processing device 3 shown in FIG.

That is, the information processing device 3 receives the identification information transmitted from the transmitters 101 to 103 attached to the monitoring target and the radio waves received by the

receivers

10 and 11 as the received radio wave intensity, and also outputs the image of the camera 1 An estimated radio wave intensity based on the image coordinates is searched from the data input unit 31 into which the image coordinates and the image acquisition time are input, and a reference database 2220 storing the reference received radio wave intensity for each work. A determination unit 32 for determining whether the received radio wave intensity is included in the allowable range, and when it is determined that the received radio wave intensity is included in the allowable range, the received radio wave intensity, identification information, image coordinates, and image acquisition time are recorded. A data recording unit 33, a data storage unit 34 for storing data of the data recording unit 33, a received radio field intensity, identification information, and image location stored in the data storage unit 34. , And a analysis unit 36 for analyzing the motion of the monitoring target by using the image acquisition time. Further, a reference data creation unit 35 for creating the reference database 2220 is provided.

The result analyzed by the analysis unit 36 is displayed on the display unit 37, and the movement of the monitoring target can be grasped.
Also, if a transmitter is attached to, or attached to, the mobile trolley 104 used at the production site, the movement history can be similarly grasped as work content. Further, the movement history of the product 105 to be produced can be similarly grasped as the work content.

When a plurality of pieces of identification information are acquired from one monitoring frame, the received radio field intensity, the identification information, the image coordinates, and the image acquisition time are recorded separately for each piece of the identification information. I can understand. Alternatively, the identification information having the largest number of appearances, that is, the identification information having the largest number of recordings may be determined to be a monitoring target, and only the identification information and the associated received radio wave intensity, image coordinates, and image acquisition time may be recorded. You may rank and record only the higher rank. By narrowing down the monitoring targets, the data capacity can be reduced.

The data of the selected monitoring target is recorded in a monitoring target database (not shown).
As described above, according to the present embodiment, the received radio wave intensity received by the receiver is based on the image coordinates of the image of the camera, and the allowable range of the estimated radio wave intensity estimated from the reference database created as the reference data. If this is true, the image coordinates, image acquisition time, and identification information related to the received radio wave intensity are stored, and by using the stored data, the resolution of the camera, the person carrying the transmitter, or the The movement of the monitoring target can be accurately grasped regardless of the state.

Embodiment 2 FIG.
A specific processing example for specifying the position information of the worker in the image of the camera 1 will be described with reference to the drawings. FIG. 4 is a schematic configuration diagram illustrating the information processing system according to the second embodiment of the present disclosure, and shows an image of the camera 1 in the work area 6.

In the image 7 of the work area 6 acquired by the camera 1, when a plurality of workers carrying the transmitter are working (FIG. 4A), the worker to be monitored is specified from the image 7. (FIG. 4B).

(4) When the worker selected as the monitoring target among the specified workers carries a transmitter, for example, the worker can be determined to be A from the identification information transmitted from the transmitter and received by the receiver. The

receivers

10 and 11 receive the radio wave transmitted from the transmitter carried by the selected worker as the received radio wave intensity, and give a time to the radio wave intensity data. Further, it is determined which transmitter the selected monitoring target carries, based on the identification information transmitted from the transmitter. The radio wave intensity data to which the identification information is added is stored in the radio wave intensity database 2210.

The received radio wave intensity of the selected worker is compared with the estimated radio wave intensity obtained from the reference database 2220. If the received radio wave intensity is within the allowable range of the estimated radio wave intensity, the received radio wave intensity is stored in the data group record database. 2230.

誰 The worker can be identified by the identification information, and the position information can be grasped from the image coordinates of the image 7 of the camera 1 and the image acquisition time.

In addition, the other workers specified in the image 7 can be similarly selected and processed. In the case where monitoring targets are mixed, an operator who is separated by each identification information and has a large number of appearances is determined to be a monitoring target, and is recorded in a monitoring target database (not shown).

Since the received radio wave intensity of the selected worker is stored in the data group record database 2230, the received radio wave intensity may be used as the position information.

If the worker selected in the image 7 does not carry the transmitter, it can be determined that the worker does not carry the data because the receiver cannot acquire the data.

Further, in FIG. 4, the movements of the mobile trolley 104 and the product 105 can be similarly grasped.
As described above, according to the present embodiment, the estimated radio wave intensity estimated at the image coordinates of the image of the camera from the reference database created as the reference data is compared with the received radio wave intensity received by the receiver, and the allowable radio wave intensity is compared. By accumulating the relevant image coordinates, image acquisition time, and identification information from the received radio wave intensity corresponding to the range, using these data, regardless of the resolution of the camera, the state of the person carrying the transmitter or the state of the thing The movement of the monitoring target can be accurately grasped.

Embodiment 3 FIG.
An example in which data acquired by the information processing device 3 is analyzed and displayed is shown. FIG. 5 is a schematic diagram of a display example of the information processing system according to the third embodiment of the present disclosure.

For example, FIG. 5A shows the work date and work time for each worker in a specific work using the data stored in the data storage database 2240, and the work time for each worker can be grasped. . FIG. 5B shows the average operation time of operations 1 to 3. It is possible to know the time required for the work, which can be used for work improvement.

It is also possible to analyze the details of the work content from the movement history of the monitoring target. For example, in a welding operation, the operation of turning on and off the power of the welding device can be grasped.

Embodiment 4 FIG.
Next, steps for specifically implementing the first to third embodiments of the present disclosure will be described.

FIG. 6 is a flowchart illustrating an information processing step according to the fourth embodiment of the present disclosure. The steps in which the

receivers

10 and 11 receive radio waves transmitted from the transmitters 101 to 103 as received radio field intensities, record the identification information of the transmitters 101 to 103 and the image acquisition time, and make the radio field intensity database 2210 are shown. Things.

The relationship between the distance between the

receivers

10 and 11 and the

transmitters

101 and 103 and the received signal strength will be described with reference to FIG.

For example, as shown in FIG. 7A, the received radio wave intensity (radio wave intensity in the graph) from the transmitter 101 received by the receiver 10 near the worker A is the received radio wave received by the receiver 11 far from the worker. It is larger than the strength (FIG. 7B).

In addition, the received radio wave intensity in FIG. 7B increases as the interval between the

receivers

10 and 11 and the

transmitters

101 and 103 decreases, and decreases as the interval increases. This corresponds to the distance between the

receivers

10 and 11.

FIG. 8 is a flowchart illustrating an information processing step according to the fourth embodiment of the present disclosure. In ST1201, from the image time database 2200 acquired and stored by the camera 1, the image data, that is, the image coordinates of the camera image, and the time data, that is, the image acquisition time, are acquired. Then, in ST1202, an object is specified from the image data.

In ST1203, a monitoring target (monitoring target) is selected from the target objects specified from the image data. In ST1204, an estimated radio field intensity is obtained from the position information of the monitoring target in the image data and the reference database 2220.

Here, a method of creating the reference database 2220 will be described with reference to FIG.
For example, the worker R performs each of the operations performed by the workers A to C, and the identification information and the reference reception radio wave intensity transmitted from the transmitter 106 carried by the worker R are received by the

receivers

10 and 11. The reference data in which these are associated with the image and the time are stored in the reference database 2220.

In ST2201, image data and time data are obtained from the image time database 2200 obtained from the camera 1. Then, in step ST2202, the worker R is specified from the image data, and in step ST2203, the coordinates as the position information of the worker R are obtained from the image data.

In ST2204, a data group transmitted from the transmitter near the time data and matching the preset transmitter number (here, the transmitter number of the transmitter 106 carried by the worker R) is acquired from the radio field intensity database 2210. In ST2205, representative data is obtained from the data group.

In ST2206, the coordinates and the representative data are recorded in the reference database 2220. Then, in ST2207, image time database 2200 confirms whether or not all processes have been performed, and in the case of “No”, returns to ST2201. If ST2207 is "yes", the process ends.

{Circle around (2)} In this way, the created reference database 2220 stores the received radio wave intensity, the coordinates of the image, and the time, which are the references of the work, in association with each other.

Next, in ST1205 of FIG. 8, a data group near time data is acquired from the radio field intensity database 2210, and one data is selected from the data group in ST1207. When the second or subsequent information processing is performed, since other data is left in the data group recording database 2230, all data in the data group recording database 2230 is deleted in ST1206 after ST1205.

In ST1208, it is determined whether or not the received radio wave intensity of the data falls between a predetermined “estimated radio wave intensity−α” and “estimated radio wave intensity + β” (permissible range). Here, α and β are predetermined thresholds.

If ST1208 is “yes”, data identification information is recorded in data group recording database 2230 in ST1209. After the process of ST1209 ends, it is determined whether all the data of the data group has been processed in ST1210. If ST1208 is "No", the process of ST1209 is skipped, and it is determined whether all the data of the data group has been processed in ST1210.

場合 If ST1210 is “No”, the process returns to ST1207 and repeats the process. If ST1210 is "yes", in ST1211, it is determined whether or not the number of data recorded in the data group recording database 2230 is equal to or larger than a threshold γ. Here, γ is a predetermined threshold.

If ST1211 is "No", the thresholds α and β are increased in ST1215, the number of data between “estimated radio field intensity−α” and “estimated radio field intensity + β” is increased, and the process returns to ST1206 to perform processing again. Is carried out. If ST1211 is "yes", in ST1212, the identification information with the largest number of appearances, that is, the identification information with the largest number of times of recording, is selected from the identification information recorded in the data group recording database 2230, and determined as the monitoring target. The monitoring object database records the time data, the monitoring object, and the transmitter identification information. By this operation, it is possible to exclude a worker or the like who is not a monitoring target and has passed by accident.

In ST1213, it is determined whether or not all the objects in the image data for which information processing is to be performed have been processed. If ST1213 is "Yes", the process ends. If ST1213 is "No", the process returns to ST1203 and the process is performed again.

Here, the process shown in FIG. 8 indicates a process at a specific time, and the same process is performed at another time. In the processing shown in FIG. 8, the processing at the time of occurrence of an error is omitted. In practice, it is necessary to perform error processing for possible errors, for example, when the database cannot be written, or when the number of data does not exceed the threshold γ even if the thresholds α and β are increased.

FIG. 10 is a flowchart illustrating an information processing step according to the fourth embodiment of the present disclosure. In FIG. 10, components denoted by the same reference numerals as those in FIG. 8 indicate the same or corresponding components, and description thereof will be omitted.

FIG. 8 is different from FIG. 8 in the configuration in the case where “No” is determined in ST1211.
If “No” is determined in ST1211, it is determined in ST1216 whether thresholds α and β are equal to or smaller than thresholds αmax and βmax, that is, whether the allowable range of estimated radio wave intensity is the maximum value. Here, αmax and βmax are predetermined thresholds. If ST1216 is “yes”, threshold α and threshold β are increased in ST1215, as in Embodiment 1, and the process is continued. If ST1216 is "No", that is, if the allowable range of the estimated radio wave intensity has reached the maximum value, "No radio wave transmission identification information" is recorded in the monitoring target database together with the time data and the monitoring target in ST1217, and the monitoring is performed. Judge that the target does not exist.

Next, in ST1213, it is determined whether or not all the objects for which information processing of image 7 needs to be performed have been processed, and the processing is performed in the same manner as in the first embodiment.

According to the above-described processing, if the received radio wave intensity received by the receiver falls within the allowable range of the estimated radio wave intensity estimated from the reference database created as the reference data based on the image coordinates of the image of the camera, this reception Accumulates image coordinates, image acquisition time, and identification information related to radio wave intensity and uses these accumulated data to accurately monitor regardless of the resolution of the camera, the state of the person carrying the transmitter, or the state of the object. Can understand the movement of the target.

Also, among the workers and objects specified in ST1202, it is possible to determine a worker or object that does not hold the transmitter or does not exist nearby. As a result, for example, a security function for determining a suspicious person and displaying a warning, and a function for determining a person who is not assumed to be in the work area to determine that abnormal work is occurring and displaying the function are provided. be able to.

Embodiment 5 FIG.
Other processing steps according to the information processing system of the present disclosure will be described.

FIG. 11 is a flowchart illustrating an information processing process according to the fifth embodiment of the present disclosure. In the process shown in FIG. 11, after the

receivers

10 and 11 receive the radio waves of the transmitters 101 to 103 as the received radio wave intensity, in ST1106, the average value of the radio wave intensity (received radio wave intensity), the identification information, and the time are recorded. , A radio field intensity database 2210. 11, the components denoted by the same reference numerals as those in FIG. 6 indicate the same or corresponding components, and the description thereof will be omitted.

Here, instead of recording all the radio wave intensity data, an average value, a standard deviation value, and a variance value of the radio wave intensity may be recorded. Thereby, the capacity of the radio field intensity database 2210 can be reduced.

FIG. 12 is a flowchart illustrating another information processing step according to the fifth embodiment of the present disclosure. In FIG. 12, radio waves of the transmitters 101 to 103 are received by the

receivers

10 and 11 as received radio wave intensity, and the average value of radio wave intensity (received radio wave intensity), identification information, and time are recorded, and the radio wave intensity database 2210 After that, in ST1107, it is determined whether or not specific radio waves are received by the

receivers

10 and 11. In FIG. 12, components denoted by the same reference numerals as those in FIG. 6 indicate the same or corresponding components, and description thereof will be omitted. FIG. 6 differs from FIG. 6 in the configuration using a specific transmitter.

In ST1107, it is determined whether or not a specific radio wave has been received. If ST1107 is "NO", the process returns to ST1101 to wait for radio waves from transmitters 101 to 103 again, and if "YES", ends the process. ST1107 may be used together with ST1104 to determine whether or not to end the system.

(4) With this, when a specific radio wave is transmitted from the transmitter, for example, it is determined that the processing is completed, and the data collection can be completed.

FIG. 13 is a flowchart illustrating another information processing step according to the fifth embodiment of the present disclosure. In FIG. 13, the radio waves of the transmitters 101 to 103 are received by the

receivers

10 and 11 as the received radio wave intensity. However, when the radio waves are not received for a certain period of time, that is, when the radio waves are not received within the reception continuation time, the reception is performed. The sensitivity is increased, and the receiving sensitivity is reduced when the frequency of receiving radio waves by the

receivers

10 and 11 is high because the receiving interval is too short. In FIG. 13, components denoted by the same reference numerals as those in FIG. 6 indicate the same or corresponding components, and description thereof will be omitted.

If ST1108 is “No”, it is determined in ST1109 whether or not radio waves have been received from transmitters 101 to 103 for δ hours. Here, δ is a predetermined threshold. If ST1108 is "yes", the process proceeds to ST1111.

If ST1109 is “Yes”, the receiving sensitivity of

receivers

10 and 11 is increased in ST1110, and the process returns to ST1101. If ST1109 is "No", the process returns to ST1108.

In ST1111, it is determined whether the interval from the previous reception is δ2 or less. Here, δ2 is a threshold value set in advance. If ST1111 is "yes", the receiving sensitivity of the receiver is reduced in ST1112, and the process proceeds to ST1103. If ST1111 is "No", the process proceeds to ST1103. In the radio field intensity database 2210, the received radio field intensity obtained by correcting the reception sensitivity is recorded.

This makes it possible to automatically change the received radio wave intensity of the

receivers

10 and 11 as necessary, and it is possible to reduce the power consumption of the receiver and improve the data collection success rate.

FIG. 14 is a flowchart illustrating another information processing step according to the fifth embodiment of the present disclosure. In FIG. 14, the transmission radio field intensity of the transmitters 101 to 103 is changed.

In FIG. 14, components denoted by the same reference numerals as those in FIG. 6 indicate the same or corresponding components, and the description thereof will be omitted.

If the radio waves are not received by the

receivers

10 and 11 for a certain period of time, the transmission radio field intensity may be increased, and if the reception interval is short, the transmission radio field intensity may be decreased to adjust the reception radio field intensity.

If ST1109 is “Yes”, the transmission radio field intensity of the transmitters 101 to 103 of the

receivers

10 and 11 is increased in ST1113, and the process returns to ST1101. If ST1109 is "No", the process returns to ST1108.

In ST1111, it is determined whether the interval from the previous reception is δ2 or less. If ST1111 is "yes", the transmission radio field intensity of transmitters 101 to 103 is reduced in ST1114, and the process proceeds to ST1103. If ST1111 is "No", the process proceeds to ST1103. In the radio wave intensity database 2210, the received radio wave intensity corrected for the change in the transmitted radio wave intensity is recorded.

(4) This makes it possible to automatically change the transmission radio field intensity of the transmitters 101 to 103 as necessary, thereby saving power of the transmitters 101 to 103 and improving the data collection success rate.

FIG. 15 is a flowchart illustrating another information processing step according to the fifth embodiment of the present disclosure. In FIG. 15, components denoted by the same reference numerals as those in FIG. 8 indicate the same or corresponding components, and a description thereof will be omitted. 8 is different from FIG. 8 in that the data in the reference database 2220 is corrected after determining “Yes” in ST1213 (ST1221).

内容 The contents of the reference database 2220 are corrected based on the position information of the transmitters 101 to 103 determined in the processing up to ST1213. For example, among the data stored in the data group recording database 2230, the average value of the data selected in ST1212 is assumed to be the estimated radio field intensity of the transmitters 101 to 103 at the position in the image, and the reference database 2220 is corrected. .

Thereby, even in an environment where the radio wave environment changes, the work content can be accurately grasped.

In the first to fifth embodiments, an example has been described in which time data is obtained from each of the camera 1 and the

receivers

10 and 11 installed at the production site. However, the function of synchronizing the time with the camera 1 and the

receivers

10 and 11 is described. May be provided. The function of synchronizing the time may be performed via the Internet. Alternatively, for example, a time synchronization server may be provided at a location connected to the information communication network such as the information processing device 3 to synchronize the time.

With this configuration, the time of the

receivers

10 and 11 and the time of the camera 1 are not shifted, and the work content can be accurately grasped.

Also, a magnet such as a permanent magnet or an electromagnet may be used instead of the transmitters 101 to 103, and a magnetic detector such as a hall sensor may be used instead of the

receivers

10 and 11. In this case, the role of the transmitter is a magnet, and the role of the receiver is a magnetic detector.

For example, it is possible to grasp position information of a monitoring target in, for example, a linear motor already equipped with a magnet.

画像 Also, the image 7 acquired from the camera 1 can be processed with a plurality of time-stamped still images, but a continuous moving image may be used. By using a moving image, when a target object is specified in ST1202, it is possible to extract a moving monitoring target from the difference between before and after. Further, in ST1202, the recognition information of the current frame can be corrected based on whether or not the object of the immediately preceding moving image frame has been recognized and whether or not the object of the next frame has been recognized.

That is, if the monitoring target is recognized in the immediately preceding frame, there is a high possibility that the monitoring target is also present in the current frame, and processing such as changing the detection sensitivity threshold based on this information becomes possible. If the image is stored as time-series data, it is possible to extract a moving monitoring target from the difference between before and after.

Thereby, it is possible to improve the identification accuracy in ST1202.
In ST1204, an example in which the estimated radio wave intensity is used from reference database 2220 has been described, but the result of performing mathematical processing on the data, such as the variance of the received radio wave intensity, the average value, the number of data per fixed time, and the like, is used as data. They may be prepared and these data may be used in ST1204.

For example, when there is a specific place where the radio wave reception intensity changes in the radio wave environment between the

receivers

10 and 11 and the transmitters 101 to 103, it is preferable to use the dispersion of the reception radio wave intensity.

In addition, as the received radio wave intensity decreases, the number of data per fixed time tends to decrease. These tendencies may be comprehensively determined and determined in ST1204. As a judgment method, there are a method of setting a threshold value, a method of storing similar tendencies in advance as a learned database through on-site tests and numerical calculations, and using this.

Thereby, even in an environment where the radio wave environment is unstable, the work content can be grasped with high accuracy.
Further, the camera 1 may be installed on a wall, a ceiling, or a pillar, for example. As long as the work area to be monitored can be photographed, the work area may be outside the work area.

Also, in FIGS. 1 and 4 (b), for convenience of explanation, the identified worker is displayed by being surrounded by a frame, but it is not necessary to actually surround the worker by a frame. What is necessary is just to hold, as data, whether or not there is a worker who has performed.

The various databases may be directly connected to the information processing device 3 and arranged, or may be connected via an information communication network and may be located elsewhere.

移動 Moreover, the mobile trolley 104 and the product 105 are shown as examples, and the monitoring target in the present disclosure may be, for example, a crane, a forklift, or an automatic carrier.

Further, the information processing device 3 may be installed outside the work area.
FIG. 5 shows an example in which the result of analyzing the movement of the monitoring target is displayed on the display 4, but the display 4 may be a display, a printer, a projector, or the like. A plurality of displays 4 may exist.

Further, the information processing apparatus 3 has an example in which the camera 1 and the

receivers

10 and 11 are installed in the work area 6 and the information processing apparatus 3 has a database, but exchanges data with an external server via a network. May go. The camera 1 may have all or a part of the functions of the information processing device 3.

Also, an example in which the information processing apparatus and the information processing system of the present disclosure are used at a production site has been described. However, the present invention is not limited to the production site, and may be used in other places (work areas) such as offices, buildings, schools, parks, and the like. Is also good.

Embodiment 6 FIG.
In the information processing system according to the present disclosure, it is also possible to grasp the direction of the worker.

FIG. 16 is a view of the production site in a state where the worker A faces the direction opposite to the receiver 10 and the camera 1 as viewed from vertically above. FIG. 17 is a view of the production site in a state where the worker A faces the receiver 10 and the camera 1 as viewed from vertically above.

In FIGS. 16 and 17, a transmitter 101 is attached around the left chest of the worker A. In the state shown in FIG. 16, the worker A is interposed between the receiver 10 and the transmitter 101, but in the state shown in FIG. 17, the worker A is not interposed between the receiver 10 and the transmitter 101. .

Generally, since a person has a property of hardly transmitting radio waves, when the worker A is not interposed between the receiver 10 and the transmitter 101 as shown in FIG. The intensity of the radio wave received by the receiver 10 is higher than that in the case where the worker A is interposed between the receiver 10 and the transmitter 101. By utilizing this characteristic, the position of the worker A can be specified by the camera 1 and the direction of the worker can be specified from the radio wave intensity at that position.

FIG. 18 is a flowchart illustrating an outline of a processing procedure according to the sixth embodiment of the present disclosure. The flowchart of FIG. 18 is obtained by changing the processing of ST1005 and ST1007 of FIG. 3 to ST2005 and ST2007, respectively. Other processes (processes with the same numbers as the processes shown in FIG. 3 described above) have already been described, and thus detailed description is omitted here.

(4) The information processing device 3 (reference data creation unit 35) creates a reference database that stores the reference radio field intensity for each work and the direction information of the worker at that time (ST2005). The work place (position) of the worker is associated with the reference radio wave intensity for each work. Therefore, the reference database is information including the correspondence between the reference received radio wave intensity, the position of the worker, and the direction of the worker.

Further, the information processing device 3 (data recording unit 33) determines whether or not the received radio field intensity in the radio field intensity database is within the allowable range of the estimated radio field intensity. Image coordinates, image acquisition time, and worker orientation information are recorded in the data group recording database 2230 (ST2007).

At this time, the information processing device 3 (data recording unit 33) specifies, for example, the orientation information of the worker as follows. When the data recording unit 33 determines that the received radio wave intensity is included in the allowable range, first, the position of the worker is specified from the image coordinates of the camera 1. Then, the data recording unit 33 refers to the reference database and specifies the direction of the worker corresponding to the received radio wave intensity and the position of the specified worker. Then, the data recording unit 33 records information indicating the direction of the specified worker in the data group recording database 2230 together with the received radio wave intensity, the identification information, the image coordinates, and the image acquisition time.

(4) Accordingly, even when the direction of the worker is difficult to be specified by the camera 1, the direction of the worker can be specified from the received radio wave intensity.

(Hardware configuration)
FIG. 19 is a diagram illustrating an example of a hardware configuration that executes a part of the functions of the information processing apparatus 3 according to Embodiments 1 to 6 described above. As shown in FIG. 19, the information processing device 3 includes a processor 1100, and a memory 1200 connected to the processor 1100 by a bus 1300.

The functions of some of the components of the information processing device 3 (for example, the data input unit 31, the determination unit 32, the data recording unit 33, the data storage unit 34, the reference data creation unit 35, and the analysis unit 36) Is realized by a processor 1110 such as a CPU executing a program stored in a memory 1200. Further, a plurality of processors and a plurality of memories may cooperate to execute the functions of the above components. Further, the functions of the above components may be realized by a processing circuit such as a system LSI. Further, a plurality of processing circuits may cooperate to execute the functions of the above components.

In the present disclosure, it is possible to freely combine the embodiments or appropriately modify or omit the embodiments within the scope of the disclosure.

The embodiments disclosed this time are to be considered in all respects as illustrative and not restrictive. The scope of the present disclosure is defined by the terms of the claims, rather than the description above, and is intended to include any modifications within the scope and meaning equivalent to the terms of the claims.

1 camera, 3 information processing device, 4 display device, 6 work area, 7 image, 10 11 receiver, 31 data input unit, 32 judgment unit, 33 data recording unit, 34 data storage unit, 35 reference data creation unit, 36 analysis unit, 37 display unit, 101, 102, 103, 106 transmitter, 104 mobile trolley, 105 product, 201, 202, 203 monitoring frame, 2200 image time database, 2210 radio field intensity database, 2220 reference database, 2230 data group Record database, 2240 data storage database.

Claims

A data input unit to which the identification information transmitted from the transmitter attached to the monitoring target and the radio wave received by the receiver are input as the received radio wave intensity, and the image coordinates of the camera image and the image acquisition time are input,
A determining unit that searches an estimated radio field intensity based on the image coordinates from a reference database in which a reference received radio field intensity for each operation is stored, and determines whether the received radio field intensity is within an allowable range of the estimated radio field intensity. When,
When it is determined that the received radio wave intensity is included in the allowable range, the received radio wave intensity, the identification information, the image coordinates, and a data recording unit that records the image acquisition time,
A data storage unit that stores data of the data recording unit;
An analysis unit that analyzes the movement of the monitoring target using the received radio wave intensity stored in the data storage unit, the identification information, the image coordinates, and the image acquisition time,
Information processing device provided with.
When there are a plurality of pieces of the identification information input to the data input unit, the received radio wave intensity, the identification information, the image coordinates, and the image acquisition time are recorded separately for each piece of the identification information. 2. The information processing device according to 1.
3. The information processing apparatus according to claim 2, wherein only the identification information having the largest number of records and the associated received radio field intensity, image coordinates, and image acquisition time are recorded.
The information processing apparatus according to claim 1, further comprising: a display unit that displays a result analyzed by the analysis unit.
The permissible range of the estimated radio wave intensity is expanded when the number of data of the received radio wave intensity, the identification information, the image coordinates, and the image acquisition time recorded by the data recording unit is equal to or less than a threshold. Information processing device.
The information processing apparatus according to claim 1, wherein the processing is terminated when a specific radio wave transmitted from the transmitter is received by the receiver and input to the data input unit.
The information processing apparatus according to claim 1, wherein if the radio wave is not received by the receiver within the reception continuation time, the reception sensitivity of the receiver is increased or the transmission radio wave intensity of the transmitter is increased.
2. The information processing apparatus according to claim 1, wherein when the frequency of receiving the radio wave by the receiver is high, the reception sensitivity of the receiver is reduced or the transmission radio field intensity of the transmitter is reduced.
In the reference database, in addition to the reference received radio field intensity, information indicating a correspondence relationship between the reference received radio field intensity, the position of the worker, and the orientation of the worker is stored,
The data recording unit, when it is determined that the received radio wave intensity is included in the allowable range, referring to the reference database and the position of the worker specified from the received radio wave intensity and the image of the camera. The information processing according to claim 1, further comprising: identifying a direction of the worker corresponding to the received information, and recording the identified direction of the worker together with the received radio wave intensity, the identification information, the image coordinates, and the image acquisition time. apparatus.
A transmitter attached to the monitoring target,
A receiver that receives the identification information and the radio wave transmitted from the transmitter,
A data input unit in which the identification information transmitted from the transmitter and a radio wave received by the receiver are input as received radio wave intensity, and image coordinates and an image acquisition time of a camera image are input,
A determining unit that searches an estimated radio field intensity based on the image coordinates from a reference database in which a reference received radio field intensity for each operation is stored, and determines whether the received radio field intensity is within an allowable range of the estimated radio field intensity. When,
When it is determined that the received radio wave intensity is included in the allowable range, the received radio wave intensity, the identification information, the image coordinates, and a data recording unit that records the image acquisition time,
A data storage unit that stores data of the data recording unit;
An analysis unit that analyzes the movement of the monitoring target using the reception radio field intensity stored in the data storage unit, the identification information, the image coordinates, and the image acquisition time,
A display unit that displays a result analyzed by the analysis unit,
Information processing system equipped with.
A step of acquiring a reference reception radio field intensity for each operation together with reference image coordinates and creating a reference database;
A step of recording the identification information and the received radio wave intensity transmitted from the transmitter attached to the monitoring target, and the image coordinates and the image acquisition time of the image acquired from the camera,
Comparing the estimated radio field intensity and the received radio field intensity retrieved from the reference database based on the image coordinates, and determining whether the received radio field intensity is included in an allowable range of the estimated radio field intensity,
When it is determined that the received radio wave intensity is included in the allowable range, the received radio wave intensity, the identification information, the image coordinates, and a step of recording the image acquisition time as data,
Saving the recorded data as accumulated data;
Analyzing the movement of the monitoring target using the accumulated data;
An information processing method comprising: