JP6972362B2 - Information processing equipment, information processing system, and information processing method - Google Patents

Description

The present disclosure relates to an information processing device, an information processing system, and an information processing method for grasping the movement of an operator or the like by using a transmitter / receiver.

Information processing systems have been developed in which radio waves transmitted from transmitters are periodically received by receivers in order to grasp the movements of equipment, people, and things (products, tools, etc.).

In particular, in production control, radio waves transmitted from a worker carrying a transmitter to which identification information is given are received by a receiver installed in the work area, and the worker is identified from the identification information to identify the received radio wave strength. A system for grasping the work contents is used. For example, the distance from the transmitter to the receiver can be grasped from the received radio wave strength, and the movement history of the worker can be known.

For example, Patent Document 1 discloses a technique in which a worker carries a sensor and automatically collects the position information of the worker to record the time required for the work and the break of the work.

There is also a system for checking the camera image of the work area to grasp the work time and work contents. For example, there is a technique in which a worker inputs identification information and position information of the worker in advance and monitors the worker with a camera image. Further, a technique is disclosed in which a camera recognizes a worker's face, position information is input in advance, and the work content is grasped by using a transmitter to which identification information is given (see, for example, Patent Document 2). ..

Japanese Unexamined Patent Publication No. 2009-294732 WO2017 / 013899A

The present disclosure relates to an information processing device, an information processing system, and an information processing method for grasping the movement of an operator or the like by using a transmitter / receiver.

However, in the above-mentioned information processing system, since the radio wave transmitted from the transmitter grasps the distance of the worker by the received radio wave intensity attenuated when the radio wave is received by the receiver, the position information of the worker is accurately obtained. It was difficult to grasp. Further, when there are a plurality of receivers, the receiver receives a plurality of radio waves transmitted from the transmitter, so that it may be difficult to grasp the exact position of the operator.

It is also possible to identify the worker by face recognition from the image of the camera, but a camera with high resolution is required. In addition, even if a high-resolution camera is used, if the worker faces downward or backward, the worker cannot be identified, and the worker's face is registered in advance in order to recognize the face in the image. There was also a need.

The present disclosure has been made to solve such a problem, and an object of the present invention is to provide an information processing device, an information processing system, and an information processing method capable of accurately grasping the identification information of the monitored object and the movement of the monitored object. And.

In the information processing apparatus according to the present disclosure, the identification information transmitted from the transmitter attached to the monitoring target and the radio wave received by the receiver are input as the received radio wave strength, and the image coordinates and the image acquisition time of the image of the camera are input. Search for the estimated signal strength based on the image coordinates from the data input section where is input and the reference database that stores the reference received signal strength for each work, and whether the received signal strength is within the allowable range of the estimated signal strength. A judgment unit that determines whether or not, and a data recording unit and a data recording unit that record the received signal strength, identification information, image coordinates, and image acquisition time when it is determined that the received signal strength is included in the allowable range. It is equipped with a data storage unit that stores the data of the above and an analysis unit that analyzes the movement of the monitored object using the received radio wave intensity, identification information, image coordinates, and image acquisition time stored in the data storage unit. ..

In addition, the information processing system according to the present disclosure includes a transmitter attached to the monitoring target, a receiver that receives identification information and radio waves transmitted from the transmitter, and identification information transmitted from the transmitter attached to the monitoring target. And the data input unit where the radio waves received by the receiver are input as the received radio wave strength, the image coordinates of the camera image and the image acquisition time are input, and the reference database that stores the reference received radio wave strength for each work. Searches for the estimated radio strength based on the image coordinates from, and determines whether the received radio strength is included in the permissible range of the estimated radio strength, and it is determined that the received radio strength is included in the permissible range. If so, the data recording unit that records the received radio wave strength, identification information, image coordinates, and image acquisition time, the data storage unit that stores the data of the data recording unit, and the received radio wave strength and identification stored in the data storage unit. It includes an analysis unit that analyzes the movement of the monitored object using information, image coordinates, and image acquisition time, and a display unit that displays the results analyzed by the analysis unit.

Further, the information processing method according to the present disclosure includes a process of acquiring a reference received radio field intensity for each work together with reference image coordinates and creating a reference database, and identification information and received radio wave strength transmitted from a transmitter attached to the monitored object. , And the process of recording the image coordinates and image acquisition time of the image acquired from the camera, and the estimated radio field strength searched from the reference database based on the image coordinates and the received radio wave strength are compared, and the received radio wave strength is the estimated radio field strength. The process of determining whether or not the data is included in the allowable range, and the process of recording the received radio field intensity, identification information, image coordinates, and image acquisition time as data when it is determined that the data is included in the allowable range. It is provided with a step of storing the recorded data as accumulated data and a step of analyzing the movement of the monitored object using the accumulated data.

According to the present disclosure, if the received radio field intensity received by the receiver falls within the permissible range of the estimated radio wave strength estimated from the reference database created as reference data based on the image coordinates of the image of the camera. By accumulating image coordinates, image acquisition time, and identification information related to the received signal strength and using these accumulated data, the resolution of the camera, the state of the person carrying the transmitter, or the state of the object can be accurately determined. You can grasp the movement of the monitored object.

It is a schematic block diagram which shows the information processing system which concerns on Embodiment 1 of this disclosure. It is a schematic block diagram which shows the information processing apparatus which concerns on Embodiment 1 of this disclosure. It is a flowchart which shows the outline of the processing procedure which concerns on Embodiment 1 of this disclosure. It is a schematic block diagram which shows the information processing system which concerns on Embodiment 2 of this disclosure. It is a schematic diagram of the display example of the information processing system which concerns on Embodiment 3 of this disclosure. It is a flowchart which shows the information processing process which concerns on Embodiment 4 of this disclosure. It is a figure explaining the relationship between the distance between the transmitter / receiver and the received radio wave strength which concerns on Embodiment 4 of this disclosure. It is a flowchart which shows the information processing process which concerns on Embodiment 4 of this disclosure. It is a flowchart which shows the creation process of the reference database which concerns on Embodiment 4 of this disclosure. It is a flowchart which shows the information processing process which concerns on Embodiment 4 of this disclosure. It is a flowchart which shows the information processing process which concerns on Embodiment 5 of this disclosure. It is a flowchart which shows the other information processing process which concerns on Embodiment 5 of this disclosure. It is a flowchart which shows the other information processing process which concerns on Embodiment 5 of this disclosure. It is a flowchart which shows the other information processing process which concerns on Embodiment 5 of this disclosure. It is a flowchart which shows the other information processing process which concerns on Embodiment 5 of this disclosure. It is the figure which looked at the production site with the worker facing the direction opposite to the receiver and the camera from the vertical upper side. It is the figure which looked at the production site with the operator facing the direction of a receiver and a camera from the vertical upper side. It is a flowchart which shows the outline of the processing procedure which concerns on Embodiment 6 of this disclosure. It is a figure which shows the example of the hardware composition which executes a part of the functions of an information processing apparatus.

Embodiment 1.
The information processing system for monitoring the monitoring target of the person, thing, etc. of the present disclosure will be described with reference to the figure. FIG. 1 is a schematic configuration diagram showing an information processing system according to the first embodiment of the present disclosure, and shows an example of monitoring the work contents of a worker using this information processing system. Depending on the situation, it is possible to grasp not only the worker but also the product, the movement of the mobile trolley, or a part of them.

In the information processing system shown in FIG. 1, a camera 1 overlooking the production site is installed, an image and an image acquisition time are acquired by the camera 1, and the image coordinates of the image become position information.

In addition, receivers 10 and 11 are installed in the information processing system, and these are radio waves transmitted from the mobile phones, that is, the transmitters 101 to 103 attached to the workers A, B, and C to be monitored. Is received as the received signal strength. This received radio wave strength is also the position information of the workers A to C. The transmitters 101 to 103 transmit radio waves and also transmit identification information assigned to each.

Then, one of the plurality of monitoring frames 201, 202, and 203 in the image of the camera 1 is selected, and for example, the image coordinates of the monitoring target in the selected monitoring frame 201 are used as the position information. Then, the received radio wave strength related to this position information is found, the position information is confirmed, and the worker A is determined from the identification information of the transmitter 101 that transmitted the radio wave.

FIG. 2 shows a schematic configuration diagram showing an information processing apparatus, and FIG. 3 shows a flowchart showing an outline of a processing procedure.

The information processing apparatus 3 is provided with a reference database 2220. For example, the worker R performs all the work performed by the workers A to C for reference, and the receivers 10 and 11 receive the radio wave of the transmitter 106 carried by the worker R as the reference reception radio wave strength. Reference data in which these are associated with the reference image coordinates of the image acquired by the camera 1 is stored in the reference database 2220. In the reference database 2220, the reference image coordinates of the camera and the reference radio wave reception time related to the work acquired in advance are recorded, and the reference reception radio wave intensity is stored.

For example, when the worker A to be monitored starts the work, the receivers 10 and 11 receive the identification information and the radio wave transmitted from the transmitter 101 carried by the worker A as the received radio wave strength (ST1001). Then, the radio waves received by the receivers 10 and 11 are recorded in the radio wave strength database 2210 as the received radio wave strength (ST1002). Since the radio wave is transmitted from the transmitter 101 carried by the worker A, the worker is determined to be A based on the identification information received by the receivers 10 and 11.

On the other hand, the worker A is projected on the selected monitoring frame 201, an image is acquired from the camera (ST1003), and the image coordinates of the image and the image acquisition time are recorded in the image time database 2200 (ST1004).

As described above, since the reference database storing the reference received radio field strength for each work has been created (ST1005), the reference received radio field strength stored in the reference database 2220 based on the image coordinates of the camera image of the worker A is used. Search for the estimated radio field strength (ST1006).

Whether or not the received radio field strength of the worker A is within the permissible range of the estimated radio wave strength is sequentially determined, and if it is within the permissible range, the image coordinates acquired by the camera 1 of the worker A and the image acquisition time are given and monitored. The movement of the target worker A, that is, the work content, is recorded in the data group recording database 2230 (ST1007). The data group recording database 2230 is rewritten as a buffer each time. Then, this operation is repeated, and the data recorded in the data group recording database 2230 is stored in the data storage database 2240 as stored data. Similarly, the work contents can be saved as data for the workers B and C.

These operations are performed by the data input unit 31, the determination unit 32, the data recording unit 33, the data storage unit 34, the reference data creation unit 35, and the analysis unit 36 of the information processing apparatus 3 shown in FIG.

That is, in the information processing apparatus 3, the identification information transmitted from the transmitters 101 to 103 attached to the monitoring target and the radio waves received by the receivers 10 and 11 are input as the received radio wave strength, and the image of the camera 1 is input. The estimated radio field strength based on the image coordinates is searched from the data input unit 31 in which the image coordinates and the image acquisition time are input and the reference database 2220 in which the reference received radio wave strength for each work is stored, and the received radio wave strength is the estimated radio field strength. The determination unit 32 that determines whether or not it is included in the allowable range, and if it is determined that the received signal strength is included in the allowable range, records the received signal strength, identification information, image coordinates, and image acquisition time. The data recording unit 33 to be monitored, the data storage unit 34 that stores the data of the data recording unit 33, and the received radio wave intensity, identification information, image coordinates, and image acquisition time stored in the data storage unit 34 are used for monitoring. An analysis unit 36 for analyzing motion is provided. Further, a reference data creation unit 35 for creating a reference database 2220 is provided.

The result analyzed by the analysis unit 36 is displayed on the display unit 37, and the movement of the monitored object can be grasped.
Further, if the transmitter is attached to the mobile trolley 104 used at the production site, that is, attached to the mobile trolley 104, the movement history can be similarly grasped as the work content. Further, for the product 105 to be produced, the movement history can be similarly grasped as the work content.

When a plurality of identification information is acquired from one monitoring frame, if the received radio wave intensity, the identification information, the image coordinates, and the image acquisition time are recorded separately for each identification information, the movements of the plurality of monitoring targets can be moved. I can grasp it. Further, the identification information having the largest number of appearances, that is, the largest number of recordings may be determined as a monitoring target, and only the identification information and the received radio wave intensity, image coordinates, and image acquisition time associated therewith may be recorded. You may rank and record only the top. By narrowing down the monitoring target, the data capacity can be reduced.

The selected monitored data is recorded in the monitored database (not shown).
As described above, according to the present embodiment, the received radio field intensity received by the receiver is an allowable range of the estimated radio wave strength estimated from the reference database created as reference data based on the image coordinates of the image of the camera. If applicable, the image coordinates, image acquisition time, and identification information related to this received signal strength are accumulated, and by using these accumulated data, the resolution of the camera, the person carrying the transmitter, or the object The movement of the monitored object can be grasped accurately regardless of the state.

Embodiment 2.
A specific processing example for specifying the position information of the operator in the image of the camera 1 will be described with reference to the figure. FIG. 4 is a schematic configuration diagram showing an information processing system according to the second embodiment of the present disclosure, and shows an image of the camera 1 in the work area 6.

In the image 7 of the work area 6 acquired by the camera 1, when a plurality of workers carrying the transmitter are working (FIG. 4A), the worker to be monitored is specified from the image 7. (Fig. 4 (b)).

When the worker selected as the monitoring target among the specified workers carries the transmitter, for example, the worker can be determined to be A from the identification information transmitted from the transmitter and received by the receiver. The receivers 10 and 11 receive the radio wave transmitted from the transmitter carried by the selected worker as the received radio wave strength, and add a time to obtain the radio wave strength data. In addition, which transmitter the selected monitoring target carries is determined by the identification information transmitted from the transmitter. The radio field strength data to which the identification information is added is stored in the radio field strength database 2210.

If the received signal strength of the selected worker is compared with the estimated signal strength obtained from the reference database 2220 and the received signal strength is within the permissible range of the estimated signal strength, the received signal strength is the data group recording database. Recorded at 2230.

Who the worker is can be identified by the identification information, and the position information can be grasped by the image coordinates of the image 7 of the camera 1 and the image acquisition time.

Further, the other workers specified in the image 7 can also be selected and processed in the same manner. When the monitoring targets are mixed, they are separated by each identification information, and the worker who appears frequently is judged to be the monitoring target and recorded in the monitoring target database (not shown).

Since the received radio wave strength of the selected worker is stored in the data group recording database 2230, the received radio wave strength may be used as the position information.

Further, when the worker selected in the image 7 does not carry the transmitter, the receiver cannot acquire the data, so that it can be determined that the worker does not carry the transmitter.

Further, in FIG. 4, the movements of the mobile trolley 104 and the product 105 can be grasped in the same manner.
As described above, according to the present embodiment, the estimated radio wave intensity estimated from the image coordinates of the image of the camera from the reference database created as the reference data and the received radio wave strength received by the receiver are compared and allowed. By accumulating related image coordinates, image acquisition time, and identification information from the received signal strength corresponding to the range, these data can be used regardless of the resolution of the camera, the state of the person carrying the transmitter, or the state of the object. The movement of the monitored object can be grasped accurately.

Embodiment 3.
An example of analyzing and displaying the data acquired by the information processing apparatus 3 is shown. FIG. 5 is a schematic view of a display example of the information processing system according to the third embodiment of the present disclosure.

For example, FIG. 5A shows the work day and work time for each worker in a specific work using the data stored in the data storage database 2240, and the work time for each worker can be grasped. .. Further, FIG. 5B shows the average working time of the works 1 to 3. You can know the time required for work, which can be useful for work improvement.

It is also possible to analyze the details of the work contents from the movement history of the monitored object. For example, in the welding work, it is possible to grasp the operation of turning on and off the power of the welding apparatus.

Embodiment 4.
Next, a step of specifically carrying out the first to third embodiments of the present disclosure will be described.

FIG. 6 is a flowchart showing an information processing process according to the fourth embodiment of the present disclosure. The process in which the receivers 10 and 11 receive the radio waves transmitted from the transmitters 101 to 103 as the received radio wave strength, record the identification information and the image acquisition time of the transmitters 101 to 103, and use the radio wave strength database 2210 as the radio wave strength database 2210 is shown. It is a thing.

The relationship between the distance between the receivers 10 and 11 and the transmitters 101 and 103 and the received radio wave intensity will be described with reference to FIG. 7.

For example, as shown in FIG. 7A, the received radio wave intensity from the transmitter 101 received by the receiver 10 near the worker A (radio wave strength in the graph) is the received radio wave received by the receiver 11 far from the worker. Greater than strength (FIG. 7 (b)).

Further, the received radio field intensity in FIG. 7B increases as the distance between the receivers 10 and 11 and the transmitters 101 and 103 increases, and decreases as the distance increases. Therefore, the received radio wave strength is from the transmitters 101 and 103. It corresponds to the distance between the receivers 10 and 11.

FIG. 8 is a flowchart showing an information processing process according to the fourth embodiment of the present disclosure. In ST1201, the image data, that is, the image coordinates of the image of the camera, and the time data, that is, the image acquisition time are acquired from the image time database 2200 acquired and stored by the camera 1. Then, the object is specified from the image data in ST1202.

In ST1203, a monitoring target (monitoring target) is selected from the objects specified from the image data. Then, in ST1204, the estimated radio field strength is obtained from the position information of the monitored object in the image data and the reference database 2220.

Here, a method of creating the reference database 2220 will be described with reference to FIG.
For example, the worker R performs all the work performed by the workers A to C, and the identification information and the reference received radio wave strength transmitted from the transmitter 106 carried by the worker R are received by the receivers 10 and 11. , The reference data associating these with the image and the time is stored in the reference database 2220.

In ST2201, image data and time data are acquired from the image time database 2200 acquired from the camera 1. Then, in ST2202, the worker R is specified from the image data, and in ST2203, the coordinates which are the position information of the worker R are acquired from the image data.

In ST2204, the data group transmitted from the transmitter near the time data and matching the preset transmitter number (here, the transmitter number of the transmitter 106 carried by the worker R) is acquired from the radio field intensity database 2210. Then, in ST2205, representative data is acquired from the data group.

In ST2206, the coordinates and representative data are recorded in the reference database 2220. Then, in ST2207, it is confirmed whether or not the image time database 2200 has processed all, and if "No", the process returns to ST2201. If ST2207 is "Yes", the process ends.

In the reference database 2220 created in this way, the received radio field intensity, the coordinates of the image, and the time, which are the reference of the work, are stored in association with each other.

Next, in ST1205 of FIG. 8, a data group near the time data is acquired from the radio field intensity database 2210, and one data is selected from the data group in ST1207. When the second and subsequent information processing is performed, since other data is left in the data group recording database 2230, all the data in the data group recording database 2230 is deleted by ST1206 after ST1205.

In ST1208, it is determined whether or not the received radio field intensity of the data is within the predetermined range of "estimated radio wave strength-α" and "estimated radio wave strength + β" (allowable range). Here, α and β are predetermined threshold values.

If ST1208 is "Yes", the data identification information is recorded in the data group recording database 2230 in ST1209. After the processing of ST1209 is completed, it is determined whether or not all the data of the data group has been processed by ST1210. If ST1208 is "No", the processing of ST1209 is skipped, and it is determined whether or not all the data of the data group has been processed by ST1210.

If ST1210 is "No", the process returns to ST1207 and the process is repeated. When ST1210 is "Yes", ST1211 determines whether the number of data recorded in the data group recording database 2230 is equal to or greater than the threshold value γ. Here, γ is a predetermined threshold value.

When ST1211 is "No", the threshold value α and the threshold value β are increased in ST1215, the number of data falling between "estimated radio field intensity -α" and "estimated radio wave strength + β" is increased, and the process returns to ST1206 and is processed again. To carry out. When ST1211 is "Yes", in ST1212, the identification information recorded in the data group recording database 2230 with the largest number of appearances, that is, the most frequently recorded identification information is selected and determined as a monitoring target. Time data, monitored objects, and transmitter identification information are recorded in the monitored object database. By this operation, it is possible to exclude workers who happened to pass by, which are not objects to be monitored.

In ST1213, it is determined whether or not all the objects that must be processed in the image data have been processed. If ST1213 is "Yes", the process ends. If ST1213 is "No", the process returns to ST1203 and the process is performed again.

Here, the process shown in FIG. 8 indicates a process at a specific time, and the same process is performed at another time. Further, in the process shown in FIG. 8, the process when an error occurs is omitted. Actually, it is necessary to perform error processing for an error that may occur, for example, when the database cannot be written, or when the number of data does not exceed the threshold value γ even if the threshold values α and β are increased.

FIG. 10 is a flowchart showing an information processing process according to the fourth embodiment of the present disclosure. In FIG. 10, those having the same reference numerals as those in FIG. 8 indicate the same or corresponding configurations, and the description thereof will be omitted.

The configuration is different from that in FIG. 8 when it is determined as "No" in ST1211.
When "No" is determined in ST1211, it is determined in ST1216 whether the threshold values α and β are equal to or less than the threshold values αmax and βmax, that is, whether or not the allowable range of the estimated radio wave strength is the maximum value. Here, αmax and βmax are predetermined threshold values. When ST1216 is "Yes", the threshold value α and the threshold value β are increased in ST1215 as in the first embodiment, and the process is continued. When ST1216 is "No", that is, when the allowable range of estimated radio field strength reaches the maximum value, ST1217 records "No radio wave transmission identification information" together with the time data and the monitored object in the monitored object database and monitors it. Judge that the target does not exist.

Next, it is determined in ST1213 whether all the objects for which information processing of the image 7 must be processed have been processed, and the processing is performed in the same manner as in the first embodiment.

If the received radio field intensity received by the receiver by the above processing falls within the allowable range of the estimated radio wave strength estimated from the reference database created as reference data based on the image coordinates of the image of the camera, this reception By accumulating image coordinates, image acquisition time, and identification information related to radio wave strength and using these accumulated data, accurate monitoring is performed regardless of the resolution of the camera and the state of the person or object carrying the transmitter. You can grasp the movement of the target.

Further, among the workers and objects specified by ST1202, it is possible to identify workers and objects that do not hold the transmitter or do not exist nearby. As a result, for example, it has a crime prevention function that identifies a suspicious person and displays a warning, and a function that identifies a person who is not expected to be in the work area and determines that out-of-steady work is occurring and displays it. be able to.

Embodiment 5.
Other processing steps related to the information processing system of the present disclosure will be described.

FIG. 11 is a flowchart showing an information processing process according to the fifth embodiment of the present disclosure. In the step shown in FIG. 11, after the receivers 10 and 11 receive the radio waves of the transmitters 101 to 103 as the received radio wave strength, the average value of the radio wave strength (received radio wave strength), the identification information, and the time are recorded in ST1106. , Radio field strength database 2210. In FIG. 11, those with the same reference numerals as those in FIG. 6 indicate the same or corresponding configurations, and the description thereof will be omitted.

Here, instead of recording all the radio field strength data, the average value, standard deviation value, dispersion value, etc. of the radio wave strength may be recorded. As a result, the capacity of the radio field intensity database 2210 can be reduced.

FIG. 12 is a flowchart showing another information processing process according to the fifth embodiment of the present disclosure. In FIG. 12, the radio waves of the transmitters 101 to 103 are received by the receivers 10 and 11 as the received radio wave strength, and the average value of the radio wave strength (received radio wave strength), the identification information, and the time are recorded in the radio wave strength database 2210. After that, it is determined whether or not a specific radio wave has been received by the receivers 10 and 11 in ST1107. In FIG. 12, those having the same reference numerals as those in FIG. 6 indicate the same or corresponding configurations, and the description thereof will be omitted. The configuration using a specific transmitter is different from that of FIG.

It is determined whether or not a specific radio wave is received in ST1107. If ST1107 is "No", the process returns to ST1101, the radio waves from the transmitters 101 to 103 are waited for again, and if "Yes", the process is terminated. ST1107 may be used in combination with ST1104 for determining whether the system is terminated.

As a result, when a specific radio wave is transmitted from the transmitter, for example, it is determined that the processing is completed, and the data collection can be terminated.

FIG. 13 is a flowchart showing another information processing process according to the fifth embodiment of the present disclosure. In FIG. 13, the receivers 10 and 11 receive the radio waves of the transmitters 101 to 103 as the received radio wave strength, but receive the radio waves when the radio waves are not received for a certain period of time, that is, when the radio waves are not received within the reception continuation time. Increase the sensitivity and decrease the reception sensitivity when the receivers 10 and 11 receive radio waves frequently because the reception interval is too short. In FIG. 13, those with the same reference numerals as those in FIG. 6 indicate the same or corresponding configurations, and the description thereof will be omitted.

When ST1108 is "No", it is determined in ST1109 whether or not radio waves are received from the transmitters 101 to 103 for δ hours. Here, δ is a predetermined threshold value. If ST1108 is "Yes", the process proceeds to ST1111.

If ST1109 is "Yes", ST1110 increases the reception sensitivity of receivers 10 and 11 and returns to ST1101. If ST1109 is "No", the process returns to ST1108.

In ST1111, it is determined whether the interval from the previous reception is δ2 or less. Here, δ2 is a preset threshold value. If ST1111 is "Yes", the reception sensitivity of the receiver is lowered in ST1112, and the process proceeds to ST1103. If ST1111 is "No", the process proceeds to ST1103. Further, the received radio field intensity corrected for the receiving sensitivity is recorded in the radio wave strength database 2210.

As a result, the received radio wave strength of the receivers 10 and 11 can be automatically changed as needed, and the power saving of the receiver and the success rate of data collection can be improved.

FIG. 14 is a flowchart showing another information processing process according to the fifth embodiment of the present disclosure. In FIG. 14, the transmission radio field strength of the transmitters 101 to 103 is changed.

In FIG. 14, those with the same reference numerals as those in FIG. 6 indicate the same or corresponding configurations, and the description thereof will be omitted.

If the receivers 10 and 11 do not receive radio waves for a certain period of time, the transmitted radio wave strength may be increased, and if the reception interval is short, the transmitted radio wave strength may be lowered and the received radio wave strength may be adjusted.

If ST1109 is "Yes", ST1113 increases the transmission radio field strength of the transmitters 101 to 103 of the receivers 10 and 11, and returns to ST1101. If ST1109 is "No", the process returns to ST1108.

In ST1111, it is determined whether the interval from the previous reception is δ2 or less. If ST1111 is "Yes", ST1114 lowers the transmission radio field strength of the transmitters 101 to 103 and proceeds to ST1103. If ST1111 is "No", the process proceeds to ST1103. Further, the radio wave strength database 2210 records the received radio wave strength corrected for the change in the transmitted radio wave strength.

As a result, the transmission radio wave strength of the transmitters 101 to 103 can be automatically changed as needed, and the power saving of the transmitters 101 to 103 and the success rate of data collection can be improved.

FIG. 15 is a flowchart showing another information processing process according to the fifth embodiment of the present disclosure. In FIG. 15, those having the same reference numerals as those in FIG. 8 indicate the same or corresponding configurations, and the description thereof will be omitted. FIG. 8 is different from the configuration in which the data in the reference database 2220 is corrected (ST1221) after the determination is “yes” in ST1213.

The contents of the reference database 2220 are modified based on the position information of the transmitters 101 to 103 determined in the processing up to ST1213. For example, among the data stored in the data group recording database 2230, the average value of the data selected in ST1212 is assumed to be the estimated radio field strength of the transmitters 101 to 103 at the positions in the image, and the reference database 2220 is modified. ..

This makes it possible to accurately grasp the work contents even in an environment where the radio wave environment changes.

In the first to fifth embodiments, the time data is acquired from each of the cameras 1 and the receivers 10 and 11 installed at the production site, but the function of synchronizing the time with the camera 1 and the receivers 10 and 11 is shown. May have. The function of synchronizing the time may be performed via the Internet, or for example, a time synchronization server may be prepared at a place connected to an information communication network such as an information processing device 3 to synchronize the time.

With this configuration, the times of the receivers 10 and 11 and the camera 1 do not deviate from each other, and the work contents can be grasped accurately.

Further, a magnet such as a permanent magnet or an electromagnet may be used instead of the transmitters 101 to 103, and a magnetic detector such as a hall sensor may be used instead of the receivers 10 and 11. In this case, the role of the transmitter is played by the magnet, and the role of the receiver is played by the magnetic detector.

For example, it is possible to grasp the position information of the monitoring target in, for example, a linear motor already equipped with a magnet.

Further, the image 7 acquired from the camera 1 can be processed by a plurality of still images to which a time is given, but a continuous moving image may be used. By making a moving image, when specifying an object in ST1202, it is possible to extract a moving monitoring target from the difference before and after. Further, in ST1202, the recognition information of the current frame can be modified based on whether or not the object of the immediately preceding moving image frame is recognized and whether or not the object of the next frame is recognized.

That is, if the monitoring target is recognized in the immediately preceding frame, it is highly likely that it also exists in the current frame, and processing such as changing the detection sensitivity threshold value based on this information becomes possible. If the image is saved as time-series data, it is possible to extract a moving monitoring target from the difference before and after.

This makes it possible to improve the identification accuracy in ST1202.
Further, in ST1204, an example of using the estimated radio field strength from the reference database 2220 is shown, but the result of performing mathematical formula processing on the data such as the variance of the received radio wave strength, the average value, and the number of data per fixed time is used as data. These data may be prepared and used in ST1204.

For example, when there is a specific place where the radio wave reception intensity changes in the radio wave environment between the receivers 10 and 11 and the transmitters 101 to 103, it is preferable to use the dispersion of the received radio wave strength.

Further, when the received radio wave intensity becomes small, the number of data per fixed time tends to decrease. These tendencies may be comprehensively judged and judged by ST1204. As a method of judgment, there are a method of setting a threshold value, a method of accumulating similar tendencies as a learned database in advance by on-site tests and numerical calculations, and using this.

This makes it possible to accurately grasp the work contents even in an environment where the radio wave environment is unstable.
Further, the camera 1 may be installed on, for example, a wall, a ceiling, or a pillar. As long as the work area to be monitored can be photographed, it may be outside the work area.

Further, in FIGS. 1 and 4B, for convenience of explanation, the specified worker is displayed by enclosing it in a frame, but it is not necessary to actually enclose it in a frame, and it is specified in which area in the information processing apparatus 3. It suffices to keep as data whether or not there is a worker who has done so.

Further, various databases may be directly connected to the information processing apparatus 3 and arranged, or may be connected via an information communication network and may be located at another location.

Further, the mobile trolley 104 and the product 105 are displayed as examples, and the monitoring target in the present disclosure may be, for example, a crane, a forklift, or an automatic guided vehicle.

Further, the information processing device 3 may be installed outside the work area.
Further, although FIG. 5 shows an example of displaying the result of analyzing the movement of the monitored object on the display device 4, the display device 4 may be a display, a printer, a projector, or the like. There may be a plurality of display devices 4.

Further, the information processing apparatus 3 has shown an example in which the camera 1 and the receivers 10 and 11 are installed in the work area 6 and the information processing apparatus 3 has a database, but data is exchanged with an external server via a network. You may go. The camera 1 may be provided with all or part of the functions of the information processing apparatus 3.

In addition, although an example of using the information processing device and information processing system of the present disclosure at a production site is shown, it is used not only at the production site but also at other places (work areas) such as offices, buildings, schools, parks, and situations. May be good.

Embodiment 6.
In the information processing system of the present disclosure, it is also possible to grasp the orientation of the worker.

FIG. 16 is a view of the production site in a state where the worker A faces the direction opposite to the receiver 10 and the camera 1 from above vertically. FIG. 17 is a view of the production site in a state where the worker A faces the receiver 10 and the camera 1 from above.

In FIGS. 16 and 17, a transmitter 101 is attached around the left chest of the worker A. In the state shown in FIG. 16, the worker A is interposed between the receiver 10 and the transmitter 101, but in the state shown in FIG. 17, the worker A is not interposed between the receiver 10 and the transmitter 101. ..

In general, since a person has a property that it is difficult for a person to pass radio waves, when the worker A is not interposed between the receiver 10 and the transmitter 101 as shown in FIG. 17, the receiver 10 is shown in FIG. The radio wave strength received by the receiver 10 is stronger than that in the case where the worker A is interposed between the transmitter 101 and the transmitter 101. By utilizing this characteristic, it is possible to specify the position of the worker A by the camera 1 and to specify the direction of the worker from the radio wave strength at the position.

FIG. 18 is a flowchart showing an outline of the processing procedure according to the sixth embodiment of the present disclosure. In the flowchart of FIG. 18, the processing of ST1005 and ST1007 of FIG. 3 described above is changed to ST2005 and ST2007, respectively. Since other processes (processes with the same numbers as those shown in FIG. 3 above) have already been described, detailed description thereof will be omitted here.

The information processing apparatus 3 (reference data creation unit 35) creates a reference database that stores the reference radio wave strength for each work and the orientation information of the worker at that time (ST2005). The work place (position) of the worker is associated with the reference radio wave intensity for each work. Therefore, the reference database is information including the correspondence between the reference received radio field intensity, the position of the worker, and the orientation of the worker.

Further, the information processing apparatus 3 (data recording unit 33) determines whether or not the received radio field strength of the radio wave strength database is within the permissible range of the estimated radio wave strength, and if it is within the permissible range, the received radio wave strength, the identification status, and the like. Image coordinates, image acquisition time, and worker orientation information are recorded in the data group recording database 2230 (ST2007).

At this time, the information processing apparatus 3 (data recording unit 33) specifies, for example, the orientation information of the worker as follows. When the data recording unit 33 determines that the received radio wave intensity is included in the allowable range, the data recording unit 33 first identifies the position of the operator from the image coordinates of the camera 1. Then, the data recording unit 33 refers to the reference database and specifies the orientation of the worker corresponding to the received radio wave intensity and the position of the specified worker. Then, the data recording unit 33 records the information indicating the direction of the specified worker in the data group recording database 2230 together with the received radio wave intensity, the identification information, the image coordinates, and the image acquisition time.

As a result, even when it is difficult to specify the orientation of the worker by the camera 1, it is possible to specify the orientation of the worker from the received radio wave strength.

(Hardware configuration)
FIG. 19 is a diagram showing an example of a hardware configuration that executes a part of the functions of the information processing apparatus 3 according to the above-described embodiments 1 to 6. As shown in FIG. 19, the information processing apparatus 3 includes a processor 1100 and a memory 1200 connected to the processor 1100 by a bus 1300.

The functions of some of the components of the information processing unit 3 (for example, data input unit 31, judgment unit 32, data recording unit 33, data storage unit 34, reference data creation unit 35, analysis unit 36) , CPU and other processors 1110 execute the program stored in the memory 1200. Further, a plurality of processors and a plurality of memories may be linked to execute the functions of the above components. Further, the functions of the above components may be realized by a processing circuit such as a system LSI. Further, a plurality of processing circuits may cooperate to execute the functions of the above components.

In this disclosure, it is possible to freely combine the embodiments and to modify or omit the embodiments as appropriate within the scope of the disclosure.

It should be considered that the embodiments disclosed this time are exemplary in all respects and not restrictive. The scope of this disclosure is set forth by the claims rather than the description above and is intended to include all modifications within the meaning and scope of the claims.

1 camera, 3 information processing device, 4 display, 6 work area, 7 image, 10, 11 receiver, 31 data input unit, 32 judgment unit, 33 data recording unit, 34 data storage unit, 35 reference data creation unit, 36 Analysis unit, 37 Display unit, 101, 102, 103, 106 transmitter, 104 mobile trolley, 105 products, 201, 202, 203 monitoring frame, 2200 image time database, 2210 radio field strength database, 2220 reference database, 2230 data group Recording database, 2240 data storage database.

Claims (11)

The identification information transmitted from the transmitter attached to the monitoring target and the radio wave received by the receiver are input as the received radio wave strength, and the image coordinates of the camera image and the image acquisition time are input to the data input unit.
Reference unit for each work A determination unit that searches the estimated radio field strength based on the image coordinates from the reference database that stores the received radio field strength, and determines whether or not the received radio field strength is within the allowable range of the estimated radio field strength. When,
When it is determined that the received radio wave intensity is included in the permissible range, the data recording unit for recording the received radio wave intensity, the identification information, the image coordinates, and the image acquisition time.
The data storage unit that stores the data of the data recording unit and the data storage unit
An analysis unit that analyzes the movement of the monitored object using the received radio wave intensity, the identification information, the image coordinates, and the image acquisition time stored in the data storage unit.
Information processing device equipped with. When there are a plurality of the identification information input to the data input unit, the identification information is separated and the received radio wave strength, the identification information, the image coordinates, and the image acquisition time are recorded. The information processing apparatus according to 1. The information processing apparatus according to claim 2, which records only the identification information having the largest number of records, the received radio wave intensity associated therewith, the image coordinates, and the image acquisition time. The information processing apparatus according to claim 1, further comprising a display unit that displays the result analyzed by the analysis unit. The first aspect of claim 1, wherein when the number of received radio wave strength, the identification information, the image coordinates, and the number of data at the image acquisition time recorded by the data recording unit is equal to or less than a threshold value, the allowable range of the estimated radio wave strength is expanded. Information processing equipment. The information processing apparatus according to claim 1, wherein when the specific radio wave transmitted from the transmitter is received by the receiver and input to the data input unit, the processing is terminated. The information processing device according to claim 1, wherein when the radio wave is not received by the receiver within the reception continuation time, the reception sensitivity of the receiver is increased or the transmission radio wave strength of the transmitter is increased. The information processing apparatus according to claim 1, wherein when the receiver frequently receives radio waves, the reception sensitivity of the receiver is lowered or the transmission radio wave strength of the transmitter is lowered. In addition to the reference received radio wave strength, the reference database stores information indicating the correspondence between the reference received radio wave strength, the position of the worker, and the orientation of the worker.
When the data recording unit determines that the received radio wave intensity is included in the permissible range, the data recording unit refers to the reference database and determines the received radio wave intensity and the position of the worker specified from the image of the camera. The information processing according to claim 1, wherein the direction of the worker corresponding to the above is specified, and the direction of the specified worker is recorded together with the received radio wave intensity, the identification information, the image coordinates, and the image acquisition time. Device. The transmitter attached to the monitoring target and
A receiver that receives the identification information and radio waves transmitted from the transmitter, and
A data input unit in which the identification information transmitted from the transmitter and the radio wave received by the receiver are input as received radio wave strength, and the image coordinates and image acquisition time of the camera image are input.
Reference unit for each work A determination unit that searches the estimated radio field strength based on the image coordinates from the reference database that stores the received radio field strength, and determines whether or not the received radio field strength is within the allowable range of the estimated radio field strength. When,
When it is determined that the received radio wave intensity is included in the permissible range, the data recording unit for recording the received radio wave intensity, the identification information, the image coordinates, and the image acquisition time.
The data storage unit that stores the data of the data recording unit and the data storage unit
An analysis unit that analyzes the movement of the monitored object using the received radio wave intensity, the identification information, the image coordinates, and the image acquisition time stored in the data storage unit.
A display unit that displays the results analyzed by the analysis unit, and
Information processing system equipped with. The process of acquiring the reference received signal strength for each work together with the reference image coordinates and creating a reference database,
The process of recording the identification information and received radio wave intensity transmitted from the transmitter attached to the monitoring target, the image coordinates of the image acquired from the camera, and the image acquisition time.
A step of comparing the estimated radio wave intensity searched from the reference database based on the image coordinates with the received radio wave intensity, and determining whether or not the received radio wave intensity is included in the allowable range of the estimated radio wave intensity.
When it is determined that the received radio wave intensity is included in the permissible range, the step of recording the received radio wave intensity, the identification information, the image coordinates, and the image acquisition time as data.
The process of saving the recorded data as accumulated data,
The process of analyzing the movement of the monitored object using the accumulated data, and
Information processing method with.

Images