JP7421402B2 - Person recognition system for specially equipped vehicles - Google Patents

Description

The present invention relates to a person recognition system for a specially equipped vehicle such as a garbage collection vehicle, and in particular, to a system for controlling the operation of a working device mounted on a vehicle chassis based on a determination that a person is near the working device. Regarding the human recognition system for specially equipped vehicles.

Conventionally, as an example, in a garbage collection vehicle, a garbage loading device (working device) is installed in a garbage input box provided at the rear of the vehicle, and the garbage loaded from the garbage input port into the garbage input box is loaded into the garbage input box. The waste is scraped into the garbage storage box using a rotating plate or a loading plate. In addition, in order to prevent workers who put garbage into the garbage inlet from getting caught in the garbage loading device due to inadvertence, we have developed a person recognition system that recognizes people near the garbage inlet. A device has been proposed (for example, see Patent Document 1).

In the technique described in Patent Document 1, a camera serving as an object image acquisition unit is disposed at the rear upper part of the garbage input box to photograph a predetermined area near the garbage input port. Then, the data of the image taken by the camera while the garbage loading device is in operation is sent to the image processing device. The device is stopped from operating.

Japanese Patent Application Publication No. 2018-154464

In the conventional technology described above, when performing person recognition processing, dictionary data stored in advance in a storage unit is referred to, and object image data extracted from an image taken by a camera is compared with the dictionary data. This makes it possible to more accurately recognize people near the garbage inlet. The dictionary data is obtained by photographing many images of people (for example, images of people's heads) in advance and extracting their characteristics such as size and shape using machine learning.

Conventional dictionary data is stored in a storage unit when a specially equipped vehicle is manufactured, and the same dictionary data is continuously used thereafter. Also, a single dictionary data was used for all special purpose vehicles produced. On the other hand, although it is considered effective to update the dictionary data according to the usage environment of the specially equipped vehicle, it is difficult for many specially equipped vehicles that are already used by many users to come to the service station each time. It was a hassle to update the dictionary data after receiving it.

On the other hand, if each specially equipped vehicle is equipped with a machine learning function that can update dictionary data, there will be no need for the vehicle to come to a service station every time. However, in that case, the problem is that installing a high-performance machine learning function on a specially equipped vehicle would lead to a significant increase in the cost of the specially equipped vehicle, so in reality it can only be installed on some specially equipped vehicles.

Furthermore, in conventional specially equipped vehicles, it was not possible to immediately switch to the optimal dictionary data according to the usage environment of the specially equipped vehicle, which changes due to changes in weather, time of day, work location, etc.

The present invention has been made in consideration of the above-mentioned circumstances, and provides a person recognition system for specially equipped vehicles that is capable of easily updating and switching dictionary data while suppressing cost increases. The purpose is to

The present invention constitutes means for solving the above-mentioned problems as follows. That is, the present invention provides a person recognition system for a specially equipped vehicle in which a working device is mounted on the chassis, and the specially equipped vehicle includes an object image acquisition unit capable of acquiring a moving image of an object near the working device. , a control unit capable of communicating with a management center server, and the management center server includes a storage unit that stores video data acquired by the object image acquisition unit and transmitted from the specially equipped vehicle; an image analysis unit that determines whether or not a person is in a person detection area near the working device by referring to dictionary data stored in the storage unit based on the video data stored in the work device. , the control unit controls the operation of the working device based on the determination result of the image analysis unit transmitted from the management center server , and the dictionary data is provided separately for each type of specially equipped vehicle or for each user. The management center server is characterized in that the management center server switches the dictionary data to be referred to when the image analysis unit makes a determination, based on the vehicle type information and user information of the specially equipped vehicle transmitted from the specially equipped vehicle.

According to the above configuration, the data of the moving image acquired by the object image acquisition section and the determination result of the image analysis section can be transmitted and received in real time between the specially equipped vehicle and the management center server. Furthermore, the management center server determines whether or not a person is in the person detection area near the working equipment of the specially equipped vehicle by referring to the dictionary data stored in the storage unit of the management center server. There is. As a result, the dictionary data can be stored in the management center server rather than in each specially equipped vehicle, so that the dictionary data can be collectively managed by the management center server. Therefore, the dictionary data can be updated all at once at the management center without having a specially equipped vehicle come to the service station. As a result, dictionary data can be easily updated. Furthermore, since a high-performance machine learning function for updating dictionary data need only be provided in the management center server, an increase in the cost of specially equipped vehicles can be suppressed.

In addition, the video data acquired by the object image acquisition unit and the judgment results of the image analysis unit can be exchanged in real time between the specially equipped vehicle and the management center server, so weather, time of day, work location, etc. may change. The management center server can immediately switch to and use the most suitable dictionary data according to the usage environment of the specially equipped vehicle. In particular, dictionary data can be switched multiple times even during one day's work on a specially equipped vehicle. Furthermore, even if there is a major change in the type of specially equipped vehicle being used by a user or the way in which the specially equipped vehicle is used, the management center server can immediately switch to the dictionary data that is appropriate for each type of specially equipped vehicle or usage environment for each user. be able to. In addition, the management center server can perform person recognition processing using dictionary data suitable for the vehicle type of each specially equipped vehicle or the usage environment of each user, so that erroneous recognition in person recognition processing can be suppressed and person recognition Processing accuracy can be improved and safety of workers etc. can be ensured.

In the present invention, the management center server includes a learning image extraction unit that extracts learning image data for updating the dictionary data at a predetermined timing from video data transmitted from the specially equipped vehicle; a machine learning section that creates update dictionary data by machine learning based on the learning image data extracted by the extraction section; Preferably, the dictionary data stored in the computer is updated.

According to the above configuration, the management center server can easily collect learning image data for updating dictionary data from many specially equipped vehicles used by many users. In addition, when extracting learning image data from video data acquired by the object image acquisition unit, the learning image extraction unit extracts the learning image data at a predetermined timing, so it is possible to extract the learning image data from the video data acquired by the object image acquisition unit. It is possible to extract learning image data whose type etc. have been specified. Thereby, compared to the case where learning image data is extracted at irregular timings, it is possible to narrow down the characteristics of a person to be subjected to machine learning in advance. In addition, since the learning image data extracted at a predetermined timing is saved in the storage unit, this saved learning image data is used to update the dictionary data to be suitable for the usage environment of each specially equipped vehicle. can do.

In the present invention, it is preferable that the control unit receives the determination result of the image analysis unit transmitted from the management center server only while the work device is in operation.

According to the above configuration, by avoiding unnecessary communication between the specially equipped vehicle and the management center server, the processing load on the management center server can be reduced, and more specially equipped vehicles can efficiently use the human recognition system. Can be done.

In the present invention, the working device is a garbage loading device disposed inside the garbage disposal box, and the image analysis unit is configured to determine whether the garbage disposal box is installed on the basis of video data transmitted from the specially equipped vehicle. Preferably, the device is configured to determine whether or not a person is present in the person detection area near the garbage inlet opening on the back side.

According to the above configuration, the management center server can have the dictionary data instead of having each garbage collection vehicle have the dictionary data, so the management center server can manage the dictionary data collectively. . Therefore, the dictionary data can be updated all at once at the management center without requiring a garbage collection vehicle to come to the service station. As a result, dictionary data can be easily updated. Furthermore, since a high-performance machine learning function for updating dictionary data only needs to be provided in the management center, it is possible to suppress an increase in the cost of garbage collection vehicles.

Furthermore, the management center server can immediately switch to and use the most suitable dictionary data according to the environment in which the garbage collection vehicle is used, which changes in weather, time of day, collection location, etc.

According to the person recognition system for a specially equipped vehicle according to the present invention, video data acquired by the object image acquisition section and judgment results from the image analysis section can be transmitted and received in real time between the specially equipped vehicle and the management center server. Since the management center server has dictionary data, it is possible to easily update the dictionary data. Furthermore, since a high-performance machine learning function for updating dictionary data only needs to be provided in the management center, it is possible to suppress an increase in the cost of specially equipped vehicles.

In addition, since the video data acquired by the object image acquisition unit and the judgment results of the image analysis unit can be exchanged in real time between the specially equipped vehicle and the management center server, the management center server can The server can immediately switch to and use the most suitable dictionary data.

BRIEF DESCRIPTION OF THE DRAWINGS It is a side view which shows the garbage collection vehicle equipped with the person recognition system based on embodiment of this invention. It is a rear view of the garbage collection vehicle of FIG. It is an explanatory view of operation of a garbage loading device equipped on a garbage collection vehicle. 4 is a view taken along the line X1 in FIG. 3. FIG. It is a hydraulic circuit diagram of a garbage loading device of a garbage collection vehicle. It is a schematic diagram showing a main body control section and an image processing unit of the garbage collection vehicle and their input/output states. It is a side view of a worker loading garbage bags into a garbage inlet, and is a diagram illustrating a schematic configuration of a person recognition system for a garbage collection vehicle. FIG. 3 is an example of an image taken by a camera, and is a diagram showing first to third detection areas. It is a flowchart which shows the flow of control performed by the image processing unit in image processing control performed by the person recognition system of the garbage collection vehicle. It is a flowchart which shows the flow of control performed by the management center server in image processing control performed by the person recognition system of the garbage collection vehicle. 3 is a flowchart illustrating an example of a person recognition process executed by a management center server.

A person recognition system for a specially equipped vehicle according to an embodiment of the present invention will be described with reference to the drawings. In the following, a case will be described in which the specially equipped vehicle is a garbage collection vehicle equipped with a garbage loading device (work device) mounted on the chassis. In addition, in the following description, for convenience, the front, rear, left, and right of the garbage collection vehicle may be simply referred to as "front, rear, left, and right."

A schematic configuration of a person recognition system for a garbage collection vehicle 100 according to an embodiment of the present invention is shown in FIG. Here, first, the garbage collection vehicle 100 provided in the person recognition system will be described. As shown in FIGS. 1 and 2, in the garbage collection vehicle 100, a garbage storage box 2 and a garbage input box 3 are provided on a vehicle chassis 1, and a rear opening of the garbage storage box 2 and a garbage input box 3 are provided. is in communication with the opening on the front side. Further, the garbage input box 3 is pivotally supported with respect to the garbage storage box 2 by a left-right pivot shaft 3a provided on the upper part thereof, and is tilted by a pair of left and right tilting cylinders (not shown). There is.

In addition, a roughly rectangular garbage input port 4 for inputting garbage G (see FIG. 8) is opened at a lower part of the back side of the garbage input box 3, and a tailgate 5 that can be raised and lowered is used to input the garbage. The mouth 4 is designed to be opened and closed. A switch box 6 is provided on the left side of the garbage inlet 4 for operating the garbage loading device. Furthermore, a camera unit 7 is disposed above the garbage inlet 4 so as to photograph the garbage inlet 4 and the area around it. The camera unit 7 is mounted on the upper part of the garbage input box 3 via a support member 80 that protrudes rearward from the inclined rear surface part (curved rear surface part) 3b of the garbage input box 3 disposed above the garbage input opening 4. is fixed.

More specifically, as shown in FIGS. 2 to 6, the support member 80 has a frame shape with base end portions at both left and right ends of the inclined rear surface portion 3b. The camera unit 7 is fixed to the support member 80 at a position in the left and right center of the garbage input box 3. The camera unit 7 includes a camera 71 as an object image acquisition unit capable of acquiring a moving image of an object near the garbage input port 4, a detection lamp 73 as a notification unit, and a second notification unit in operation. It has a lamp 72. In the person recognition system of the garbage collection vehicle 100, the data of the image (video) taken by the camera 71 is transmitted in real time to the management center server 200 (see FIG. 7) of the management center, and is stored in the server side storage of the management center server 200. 205.

The camera 71 is attached to the rear upper part of the garbage input box 3 at a predetermined attachment angle. Specifically, the camera 71 is mounted in a recess 74 provided at the left and right center portions of the camera unit 7 via a mounting bracket 75. The camera 71 has an imaging lens 71a, and is attached to the mounting bracket 75 of the camera unit 7 so as to be vertically adjustable around a horizontal axis so that the imaging lens 71a faces diagonally downward. On both left and right sides of the recessed portion 74 of the camera unit 7, fixing surfaces 7a are provided that are inclined rearward and upward with respect to the horizontal plane (that is, the fixed surfaces 7a are inclined upward toward the rear of the vehicle). An operating lamp 72 is attached to the left fixed surface 7a facing downward. Further, a detection lamp 73 is attached to the right fixed surface 7a facing downward. That is, the operating lamp 72 and the detection lamp 73 are provided on both the left and right sides of the camera 71 near the imaging lens 71a of the camera 71.

Next, as shown in FIG. 3, the inside of the garbage input box 3 is equipped with a garbage loading device (work device) for loading the input garbage G into the garbage storage box 2. This garbage loading device pushes the garbage loaded into the garbage input box 3 into the garbage storage box 2 which is connected in front of the garbage input box 3. The garbage storage box 2 is provided with a garbage discharge device (not shown) that discharges the contained garbage. This garbage discharge device may be configured such that, for example, the garbage storage box 2 is tilted by a dump cylinder interposed between the vehicle chassis 1 and the garbage storage box 2 to discharge the garbage, or the garbage storage box 2 is installed inside the garbage storage box 2. It is conceivable to move a discharge plate to the rear of the garbage storage box 2 using a discharge cylinder to discharge the garbage.

Next, the garbage loading device of this embodiment will be specifically explained. The garbage loading device of this embodiment is a so-called rotating plate type garbage loading device that scoops up garbage G by rotating a rotating plate (loading member) 10 and pushes it into the garbage storage box 2 by a pushing plate 20. It is configured. A rotating shaft 11 is installed in the lower part of the garbage input box 3 so as to extend in the width direction thereof, and the base end side of the rotating plate 10 is fixed to this.

In the illustrated example, a hydraulic motor 13 capable of forward and reverse rotation is connected to an end of the rotating shaft 11 via a speed reduction mechanism 12 . The rotation of the hydraulic motor 13 is torqued up by the reduction mechanism 12 and transmitted to the rotating shaft 11, and the rotating plate 10 is rotated together with the rotating shaft 11, so that its tip has a substantially semicircular arc shape in cross section. It comes to move in the front and back direction along the bottom wall of the formed garbage input box 3.

On the other hand, the push plate 20 is provided above the rotary plate 10 over the entire width direction of the garbage input box 3, and is swingable in the front and rear directions around a swing shaft 21 in the left and right direction provided at the upper part of the push plate 20. Supported. Further, the pushing plate 20 is provided with an extending portion 22 extending above the swing shaft 21, and a pushing cylinder 24 is installed between this extending portion 22 and a support pin 23 in front of the extending portion 22. The pushing plate 20 is swung back and forth by the expansion and contraction operation.

Specifically, as shown by the solid line in FIG. 3, when the pushing plate 20 is at the position where it swings the most towards the garbage storage box 2 (advance limit position), the pushing plate 20 rotates without interfering with the pushing plate 20. The plate 10 begins to rotate upward, and the pushing plate 20 swings toward the garbage inlet 4 side after a delay. Even after the pushing plate 20 swings the most towards the garbage inlet 4 side and reaches the backward limit position shown by the imaginary line in FIG. 3, the rotary plate 10 continues to rotate.

The rotary plate 10 rotating in this manner scrapes the garbage G into the garbage storage box 2 side, and once stops at a set stop position extending toward the front garbage storage box 2 side, as shown by the solid line in FIG. . Then, the pushing plate 20 swings toward the garbage storage box 2 side, and pushes the garbage G on the rotary plate 10 into the garbage storage box 2. Then, when the pushing plate 20 reaches the forward limit position again, the rotary plate 10 begins to rotate upward again.

By repeating the rotation of the rotary plate 10 and the rocking of the pushing plate 20 in synchronization with each other in this way, a garbage loading operation is performed in which the garbage G input into the garbage input box 3 is continuously loaded into the garbage storage box 2. will be held. The configuration of the hydraulic circuit and control system for operating the rotating plate 10 and the pushing plate 20 in this manner will be described later.

Inside the garbage input box 3, switches LS1 to LS4 are provided for detecting the positions of the rotating plate 10 and the pushing plate 20. Specifically, as shown in FIG. 3, switches LS1 and LS2 are turned on when the push plate 20 is at the forward limit position or the backward limit position, and switches LS1 and LS2 are turned on when the rotating plate 10 is at the set stop position. A switch LS3 that turns on when the rotary plate 10 rotates a predetermined angle in the positive direction (clockwise in FIG. 1) from its set stop position and a switch LS4 that turns off when it further rotates a predetermined angle are provided. It is being

The switches LS1 and LS2 are designed to detect a dog (not shown) provided at the end of the pivot shaft 21 of the push plate 20, and the switches LS3 to LS4 are adapted to detect a dog (not shown) provided at the end of the pivot shaft 21 of the rotating plate 10. A dog (not shown) provided at the end of the sensor is detected. Further, as these switches LS1 to LS4, for example, limit switches, photoelectric switches, proximity switches, etc. can be used. In addition, as shown by hatching in FIG. 3, the switch LS4 rotates and descends from just below the front edge (upper edge) 4a of the garbage inlet 4 to the rear thereof. This sensor detects the angular range Z up to the point closest to the rear edge (lower edge) 4b, and detects that the rotary plate 10 is operating near the garbage inlet 4. .

Furthermore, as shown in FIGS. 1 and 3, emergency stop buttons 60, 61 and an emergency stop plate 62 are provided near the garbage inlet 4 to stop the operation of the garbage loading device. There is. As shown in FIG. 1, an emergency stop button 60 (corresponding to switch SW1 in FIG. 6) is provided on the side of the switch box 6 provided on the left side of the garbage inlet 4, and as shown by the broken line in FIG. An emergency stop button 61 (corresponding to switch SW3 in FIG. 6) is provided on the right side of the garbage inlet 4. The emergency stop plate 62 is arranged below the garbage inlet 4 to turn on and off the switch SW2. Further, as shown in FIG. 2, a stop release switch 63 is provided on the rear surface of the switch box 6. This stop release switch 63 is arranged to turn on and off a momentary switch SW4, for example. The stop release switch 63 is operated when canceling an emergency stop of the garbage loading operation of the garbage loading device, which will be described later.

-Control system of garbage loading device-
Next, a control system for operating the garbage loading device will be described with reference to FIGS. 5 to 7. This control system includes a hydraulic circuit that controls the hydraulic pressure supplied to the hydraulic motor 13 of the garbage loading device, the pushing cylinder 24, etc., and a main body that outputs control signals to the electromagnetic control valves V1 and V2 provided in this hydraulic circuit. It includes a control unit PLC (programmable logic controller) and an image processing unit 9 that causes the management center server 200 to execute person recognition processing based on video data from the camera 71. The main body control unit PLC is configured to control not only the drive of the garbage loading device but also the drive of the garbage discharge device. Further, the image processing unit 9 and the main body control section PLC constitute a control device for the garbage loading device, which is a mounted object.

First, the hydraulic circuit will be explained with reference to FIG. This hydraulic circuit includes a hydraulic pump P, an oil reservoir T, an electromagnetic control valve V1 for controlling the push cylinder 24, and an electromagnetic control valve V2 for controlling the hydraulic motor 13. Note that driving force is transmitted to the hydraulic pump P by a PTO (power take-off) that takes out the power of an engine (not shown) as a driving source for driving the vehicle.

As an example, the electromagnetic control valves V1 and V2 are both 6-port, 3-position electromagnetic directional switching valves. When the solenoid SOLa is excited by the main body control unit PLC, the electromagnetic control valve V1 switches to the first communication position (upper position in FIG. 5), and transfers the hydraulic oil from the hydraulic pump P to the rod side of the pair of push cylinders 24. Supply to the oil chamber. On the other hand, when the solenoid SOLb is excited by the main body control unit PLC, the electromagnetic control valve V1 switches to the second communication position (lower position in FIG. 5) and supplies hydraulic oil to the head side oil chamber.

Then, when hydraulic oil is supplied from the electromagnetic control valve V1 to the head-side oil chamber, the pair of push cylinders 24 are extended to swing the push plate 20 forward. On the other hand, when hydraulic oil is supplied to the rod-side oil chamber, the pair of push cylinders 24 are contracted to swing the push plate 20 backward. Furthermore, when neither of the solenoids SOLa and SOLb is energized, the electromagnetic control valve V1 returns to the neutral position (center position in FIG. 5).

When the solenoid SOLc is excited, the electromagnetic control valve V2 switches to the first communication position (lower position in FIG. 5), supplies hydraulic oil to the normal rotation side oil chamber of the hydraulic motor 13, and operates the hydraulic motor 13. In addition to normal rotation, the push cylinder 24 can also be telescopically operated. On the other hand, when the solenoid SOLd is energized, the electromagnetic control valve V2 switches to the second communication position (upper position in FIG. 5), supplies hydraulic oil to the reverse side oil chamber of the hydraulic motor 13, and operates the hydraulic motor 13. Operate in reverse.

Furthermore, when neither of the solenoids SOLc and SOLd is energized, the electromagnetic control valve V2 returns to the neutral position (center position in FIG. 5). When both the electromagnetic control valves V1 and V2 are in the neutral position, the hydraulic oil returns to the oil reservoir T. In the illustrated hydraulic circuit, reference numeral V3 is a check valve, and reference numeral V4 is a relief valve for setting the upper limit of the discharge pressure of the hydraulic pump P.

Next, the input/output state of signals of the main body control section PLC and the image processing unit 9 will be explained with reference to FIG. First, power is supplied to the main body control unit PLC by the battery BT. An ignition switch SWK of the garbage truck 100, a PTO switch (drive source switch) SWP, a relay coil R1, etc. are interposed in the current-carrying line K2 extending from the positive electrode of the battery BT to the right side of FIG. 6 to the ground line K1. ing.

In addition, the upstream end of the current-carrying line K3 is connected so as to branch from the current-carrying line K2 midway between the ignition switch SWK and the battery BT, and the contact of the relay coil R1 ( A relay switch) r1 is provided. A power lamp L is interposed in this energizing line K3, and when the relay coil R1 is energized and the contact r1 is closed, the power lamp L is turned on by energizing the energizing line K3.

In addition, the upstream end of the energizing line K4 is connected so as to branch from the energizing line K3 at a point between the contact r1 of the relay coil R1 and the power lamp L. (omitted). That is, when the contact r1 is closed, power is supplied to the main body control unit PLC via the energizing lines K3 and K4.

Furthermore, the main body control unit PLC is always energized by the energizing line K5 branching from the energizing line K4 during the garbage loading operation of the garbage loading device. In other words, the energization line K5 is a line for inputting a so-called loading continuation signal, and includes switches SW1 to SW3 that are opened and closed in response to the operation of the emergency stop buttons 60 and 61 and the emergency stop plate 62, etc. is interposed. When energization (that is, input of the loading continuation signal) is cut off by these switches SW1 to SW3, the main body control unit PLC outputs a control signal to excite the solenoids SOLa to SOLd of the electromagnetic control valves V1 and V2. It is supposed to turn off. As a result, the electromagnetic control valves V1 and V2 return to their neutral positions, and the operation of the garbage loading device is stopped.

Further, a plurality of branch lines branching from the middle of the energizing line K4 are connected, and the above-mentioned switches LS1 to LS4 are interposed in each of these branch lines. Signals from the switches LS1 to LS4 are input to the main body control unit PLC, and based on these signals, the positions of the rotating plate 10 and the pushing plate 20 of the garbage loading device, in other words, the operating status are detected. Ru.

Furthermore, in addition to switches LS1 to LS4, on the input side to the main body control unit PLC, there is a loading switch SW5 as an actuator operation unit for cyclic operation of the garbage loading device, and a garbage loading operation of the garbage loading device. Or a changeover switch SW6 for switching the dust discharge operation of the dust discharge device, a switch SW7 for tilting and opening the dust disposal box 3, a single-acting or continuous selection switch for the dust loading operation (not shown), a rotary plate 10 and a switch (not shown) that independently operates the push plate 20 and the like are also electrically connected. The changeover switch SW6 is provided at a branch position where the current conduction line K5 branches from the current conduction line K4, and the current conduction line K5 is connected to the loading side of the changeover switch SW6.

While various switches are connected to the input side as described above, the solenoids SOLa to SOLd of the electromagnetic control valves V1 and V2 mentioned above are connected to the output side of the main body control unit PLC. Then, the main body control unit PLC operates the corresponding solenoids SOLa to SOLa to operate the hydraulic motor 13, push cylinder 24, etc. according to a preset procedure based on signals input from switches SW1 to SW3, LS1 to LS4, etc. It is programmed to output to SOLd.

Specifically, when the garbage loading device operates to load garbage, when both the ignition switch SWK and the PTO switch SWP on the energizing line K2 are turned on, the relay coil R1 is excited. As a result, contact r1 of relay coil R1 is closed, and power is supplied to the main body control unit PLC through the energizing lines K3 and K4, so that the main body control unit PLC becomes operable and the solenoids SOLa to SOLd are activated as appropriate. A control signal will be output to.

In response to this control signal, the solenoids SOLa to SOLd are excited, and the positions of the electromagnetic control valves V1 and V2 are appropriately switched, thereby supplying hydraulic pressure to the hydraulic motor 13, push cylinder 24, and the like. As a result, the hydraulic motor 13, the push cylinder 24, etc. are operated, and as described above, the rotation of the rotating plate 10 and the swinging of the push plate 20 are repeated in synchronization with each other.

In detail, first, as shown by the solid line in FIG. 3, the push plate 20 is at the forward limit position and an on signal is output from the switch LS1, and the rotating plate 10 is at the set stop position and the switch LS3 is also output. When the ON signal is output, a control signal is output from the main body control unit PLC that has received the signal from the loading switch SW5, the electromagnetic control valve V2 is switched to the first communication position, and the hydraulic motor 13 is operated in the forward direction. Start. As a result, the rotary plate 10 begins to rotate upward.

When a predetermined period of time has elapsed, a control signal is output from the main body control unit PLC to the solenoid SOLa of the electromagnetic control valve V1, the electromagnetic control valve V1 is switched to the first communication position, and the push cylinder 24 starts to contract. . This causes the push plate 20 to swing toward the rear garbage inlet 4 side, and when the push plate 20 reaches the backward limit position, an on signal is output from the switch LS2.

In response to this, the main body control unit PLC stops outputting the control signal to the solenoid SOLa, so that the electromagnetic control valve V1 returns to the neutral position and the swinging of the push plate 20 is stopped. Also, while the pushing plate 20 is swinging, the rotating plate 10 continues to rotate, and the garbage G is pushed into the garbage storage box 2 side. reaches the set stop position, and an on signal is output from switch LS3.

In response to this, the main body control unit PLC stops outputting the control signal to the solenoid SOLc of the electromagnetic control valve V2, so that the electromagnetic control valve V2 returns to the neutral position and the rotation of the hydraulic motor 13 is stopped. In addition, the main body control unit PLC outputs a control signal to the solenoid SOLb of the electromagnetic control valve V1, and the electromagnetic control valve V1 is switched to the second communication position and the push cylinder 24 starts to extend, thereby causing the push plate 20 begins to swing forward.

In this way, the pushing plate 20 swinging toward the front garbage storage box 2 pushes the garbage G on the rotary plate 10 into the garbage storage box 2, and when it reaches the forward limit position, an on signal is output from the switch LS1. . In response to this, the main body control unit PLC stops outputting the control signal to the solenoid SOLb, so that the electromagnetic control valve V1 returns to the neutral position, and the pushing cylinder 24 is extended, that is, the pushing plate 20 is moved forward. The rocking stops and the series of operations ends.

Further, as shown in FIG. 6, the upstream end of the energizing line K6 is connected between the ignition switch SWK and the PTO switch SWP, and the image processing unit 9 is connected to the energizing line K6. When the ignition switch SWK is turned on, power is supplied to the image processing unit 9 via this energizing line K6. The image processing unit 9 is connected to the camera 71 of the camera unit 7 via a power supply line K7. Further, a reverse signal based on the driving operation of the vehicle is input to the image processing unit 9 via the energizing line K8.

The image processing unit 9 includes a central processing unit CPU that executes a predetermined program and performs various controls, a memory M as a storage unit that stores data used in the central processing unit CPU, and a memory M that stores instructions from the central processing unit CPU. an image output unit VOP that displays the results of image processing on a monitoring monitor 93 (see FIG. 1); a clock unit C that measures the time of the data log; and a camera control unit (not shown) that controls the camera 71. etc. are provided. Note that the upstream end of the energizing line K9 is connected so as to branch from the energizing line K3 on the upstream side of the contact r1 of the relay coil R1, and power is supplied to the timer C via this energizing line K9. be exposed. Therefore, even when the ignition switch SWK is off, the timekeeping section C is always energized.

Further, the image processing unit 9 is connected to an energizing line K10 branching from the energizing line K5. This energizing line K10 does not function during the garbage discharging operation of the garbage loading device, but serves as a loading signal line for functioning during the garbage loading operation of the garbage loading device. The energizing line K10 is connected to the loading side of the changeover switch SW6, and when the PTO switch SWP is on and the changeover switch SW6 is switched to the loading side, the loading line is connected to the image processing unit 9. The signal is now being input.

An actuation switch SWS is interposed between the image processing unit 9 and the main body control section PLC. The image processing unit 9 is provided with an output port for a person safety signal, and this output port and the actuation switch SWS are connected by an energizing line K16. Further, the image processing unit 9 is provided with an input port for a data log start signal, and this input port and the actuation switch SWS are connected by a current supply line K17. Furthermore, an energization line K15 branched from the energization line K10 is connected to the actuation switch SWS.

When the operation switch SWS is switched to the on side, the output port of the person safety signal of the image processing unit 9 and the main body control unit PLC are connected via the energization line K16 and the energization line K12.

Here, the person safety signal output from the image processing unit 9 is input to the main body control unit PLC through the energization line K16 and the energization line K12. However, if a person enters the prohibited area (second detection area Y12 described later) during the garbage loading operation of the garbage loading device, the image processing unit 9 does not output a person safety signal, and the main body control unit PLC The person safety signal is not input. The main body control unit PLC is configured to perform an emergency stop of the garbage loading operation of the garbage loading device under one of the conditions that there is no input of the person safety signal.

Moreover, when the operation switch SWS is switched to the on side, the energization line K11 branched from the energization line K6 is connected to the energization line K17. In this case, when the ignition switch SWK is turned on, the energization line K6 is energized, and a data logging start signal is sent to the image processing unit 9 through the energization line K11 and the energization line K17. Data logging is then executed using this as a trigger.

On the other hand, when the operation switch SWS is switched to the off side, the energization line K10 and the main body control unit PLC are connected via the energization line K15 and the energization line K12. In this case, when the loading line signal is being input to the image processing unit 9, the voltage from the energizing line K10 is input to the main body control unit PLC instead of the person safety signal. Therefore, regardless of whether or not a person is recognized in the prohibited area, the main body control unit PLC recognizes that the person safety signal is always input.

Note that when the operating switch SWS is switched to the off side, the energization line K17 is not energized and data logging is not started.

Further, a switch SW4 corresponding to the stop release switch 63 is interposed between the image processing unit 9 and the main body control section PLC. The switch SW4 is provided on an energization line K13 branched from the energization line K10. When the stop release switch 63 is not turned on, the switch SW4 is turned off, and no signal is input to the main body control unit PLC. On the other hand, when the stop release switch 63 is turned on, the switch SW4 is turned on, and when the loading line signal is input to the image processing unit 9, the voltage from the energized line K10 is The signal is input to the main body control unit PLC as an on signal. Note that during the garbage discharge operation of the garbage loading device, since the loading line signal is not input to the image processing unit 9, the function of the stop release switch 63 is disabled.

When the main body control unit PLC recognizes the ON signal of the stop release switch 63, even if the input of the human safety signal from the image processing unit 9 disappears, the main body control unit PLC immediately stops the garbage loading operation of the garbage loading device. It is designed not to do this.

The image processing unit 9 includes a pilot lamp (not shown) disposed around the driver's seat, an operating lamp 72 as a notification section disposed near the garbage inlet 4, and a detection lamp 73 that are electrically connected to the image processing unit 9. The lighting control is performed by the central processing unit CPU. Further, a buzzer 91 that outputs a backward warning sound is connected to the image processing unit 9, and this control is also performed by the central processing unit CPU. The low potential sides of the operating lamp 72, the detection lamp 73, and the buzzer 91 are connected to the ground line K1 via the energization line K13.

Further, as shown in FIG. 7, in the person recognition system for the garbage collection vehicle 100, the image processing unit 9 can be communicatively connected to the management center server 200. Specifically, the image processing unit 9 is connected to a first communication section 101 provided in the garbage collection vehicle 100. The first communication unit 101 is capable of communicating with a second communication unit 201 provided in the management center server 200 in the management center, for example, via a network such as the Internet. Data of a moving image photographed by the camera 71 is transmitted from the image processing unit 9 to the management center server 200 via the first communication section 101 in real time. The video data received by the management center server 200 is stored in the server-side storage unit 205. Further, the determination result of the person recognition process is transmitted from the management center server 200 to the image processing unit 9 via the first communication unit 101 in real time. The determination result of the person recognition process received by the image processing unit 9 is stored in the memory M.

In addition to the above-mentioned first communication section 101, operation switch SWS, PTO switch (drive source switch) SWP, loading switch (actuator operation section) SW5, operating lamp 72, and detection lamp 73, the garbage collection vehicle 100 includes a position An information acquisition unit 102 and the like are provided. The position information acquisition unit 102 can acquire the position information of the garbage collection vehicle 100 by receiving radio waves from a GPS satellite (not shown), for example.

The first communication unit 101 transmits to the management center server 200 the vehicle model information and user information of the garbage collection vehicle 100 stored in the memory M, the video data captured by the camera 71, and the position information acquisition unit 102. The acquired position information, loading line signal, and retreat signal are transmitted.

The management center in which the management center server 200 is installed is owned by, for example, a bodywork manufacturer that manufactures and sells the garbage collection vehicle 100 including the image processing unit 9. Note that the management center may be, for example, a business office of a collection company.

The management center server 200 includes an image analysis section 202, a learning image extraction section 203, a machine learning section 204, and the like, in addition to the second communication section 201 and the server-side storage section 205 described above. The image analysis unit 202 performs person recognition processing based on video data acquired by the camera 71 and input to the management center server 200. The learning image extraction unit 203 extracts learning image data for updating dictionary data from the video data acquired by the camera 71 and input to the management center server 200 at a predetermined timing, and stores the extracted learning image data in the server side storage unit 205. It is to be stored in The machine learning unit 204 creates update dictionary data by machine learning based on the learning image data extracted by the learning image extracting unit 203 and stores it in the server-side storage unit 205.

In the management center server 200, a machine learning unit 204 performs machine learning based on the learning image data extracted by the learning image extraction unit 203, and update dictionary data is created. Further, based on the update dictionary data created by the machine learning unit 204, the dictionary data stored in the server-side storage unit 205 is updated. When the image analysis unit 202 performs person recognition processing, the dictionary data in the server-side storage unit 205 that has been updated using the update dictionary data is referred to.

The server-side storage unit 205 stores dictionary data that is referred to during person recognition processing. The dictionary data stored in the server-side storage unit 205 includes dictionary data for loading (work) and dictionary data for retreating. Furthermore, the learning image data extracted by the learning image extraction unit 203 is stored in the server side storage unit 205. The learning image data stored in the server-side storage unit 205 includes learning image data for loading (work) and learning image data for reversing. Furthermore, the server-side storage unit 205 stores update dictionary data created by the machine learning unit 204. The update dictionary data stored in the server-side storage unit 205 includes update dictionary data for loading (work) and update dictionary data for backing.

Here, the dictionary data for loading (work) stored in the server side storage unit 205 is provided separately for each user of the garbage collection vehicle 100, and is also provided for each type of garbage collection vehicle 100. It is set up separately. Furthermore, the dictionary data for reversing stored in the server-side storage unit 205 is provided separately for each user and also for each type of vehicle used. Then, based on the vehicle type information and user information of the garbage collection vehicle 100 transmitted from the image processing unit 9, the dictionary data referred to in the determination by the image analysis section 202 is switched.

Furthermore, the server-side storage unit 205 also stores a person detection area used for person recognition processing. The person detection areas stored in the server-side storage unit 205 include a loading (work) person detection area (first detection area Y11, second detection area Y12) and a backing person detection area. area (third detection area Y2).

In this embodiment, an operating lamp 72 and a detection lamp 73 are disposed on both the left and right sides of the imaging lens 71a of the camera 71 (see FIG. 4). In principle, the operating lamp 72 is turned on (lit) when a loading line signal is input to the image processing unit 9. In principle, the detection lamp 73 is activated when a loading line signal is input to the image processing unit 9 and it is determined that a person is in the person detection area near the garbage input port 4 through person recognition processing, which will be described later. Turned on (lit).

Further, as shown in FIGS. 1, 3, and 7, a camera 71 is attached at a predetermined mounting angle at the upper part of the back of the garbage input box 3, in other words, above the garbage input port 4, and its imaging lens 71a is directed diagonally downward at the rear. Figures 7 and 8 show a person H (worker) standing near the garbage inlet 4 with his arms extended forward and loading garbage G (garbage bags in Figures 7 and 8). has been done. The camera 71 is originally used as a back camera for the driver of the garbage collection vehicle 100 to monitor the rear, and as an example is shown in FIGS. Take a photo.

The optical axis of the imaging lens 71a of the camera 71 extends in the direction along the X1 line in FIG. A nearby person H and garbage G are photographed from above. Then, based on the data of the image photographed (acquired) by the camera 71 and sent to the management center server 200, the management center server 200 detects whether a person H such as a worker is in the garbage inlet 4 and the person behind it. It is determined whether or not the vehicle is in the area (the area surrounded by a broken line and the area surrounded by a solid line in FIG. 8). Further, if the management center server 200 determines that the person H is present in the person detection area, the determination result is sent to the image processing unit 9, and the image processing unit 9 uses the determination result of the management center server 200 to It is determined whether or not to output a person safety signal to the control unit PLC. The main body control unit PLC operates the garbage loading device based on whether a person safety signal is received from the image processing unit 9 and whether or not the rotary plate 10 is detected to be in the angle range Z by the switch LS4. It is determined whether or not to stop it.

Specifically, the area for detecting people for loading is divided into a first detection area Y11 (indicated by a broken line in FIG. 8) and a second detection area Y12 (indicated by a solid line in FIG. 8). . The second detection area Y12 is provided closer to the garbage inlet 4 than the first detection area Y11, and has a function as an entry-prohibited area. For example, the first detection area Y11 is a normal area when a worker performs garbage loading work, and is set in a rectangular area near the garbage inlet 4. The second detection area Y12 is set in a rectangular area between the front edge 4a and the vicinity of the rear edge 4b of the garbage inlet 4. This second detection area Y12 is set to include at least a part of the area surrounded by the front edge 4a, rear edge 4b, left edge 4c, and right edge 4d of the garbage inlet 4. .

On the other hand, when the vehicle is reversing, the management center server 200 receives the reversing signal without receiving the loading line signal. Based on this condition, the management center server 200 sets dictionary data for reversing according to the vehicle type and user information, and detects the person H in the third detection area Y2 behind the garbage inlet 4 (FIG. 7, (indicated by a dashed line in Figure 8). If the management center server 200 determines that the person H is present in the third detection area Y2, the management center server 200 transmits the determination result to the image processing unit 9, and the image processing unit 9 notifies the driver and calls for attention. I try to do that. The third detection area Y2 is set as a rectangular area including the second detection area Y12, the first detection area Y11, and the area behind them. In addition, on the monitor 93 of the garbage collection vehicle 100, along with the image of the camera 71, a line corresponding to the outer edge of the first detection area Y11 (indicated by a broken line in FIG. 8), and a second detection area (no entry area) are displayed. A line corresponding to the outer edge of Y12 (shown as a solid line in FIG. 8) and a line corresponding to the outer edge of the third detection area Y2 (shown as a dashed-dotted line in FIG. 8) are displayed.

- Image processing routine for human recognition system -
Next, a control routine based on image processing performed by the person recognition system of the garbage collection vehicle 100 will be described with reference to the flowcharts of FIGS. 9 and 10. 9 and 10, for convenience, control executed by the image processing unit 9 shown in FIG. 9 and control executed by the management center server 200 shown in FIG. 10 are shown separately.

First, in step S1 after the start, when the operator performs an operation to switch the ACC power of the garbage collection vehicle 100 from off to on, the power of the image processing unit 9 is turned on from off. In step S2, initialization processing (startup processing) of the image processing unit 9 is performed. In step S3, when the operator performs an operation to switch the PTO switch SWP from off to on, in step S4, the central processing unit CPU of the image processing unit 9 determines whether the operation switch SWS is on. If the operation switch SWS is on (YES), the process proceeds to step S5, and if the operation switch SWS is off (NO), the process proceeds to step S30. In step S30, the image processing unit 9 does not output an on signal for the operating lamp 72 and an off signal for the detection lamp 73, and proceeds to step S17.

In step S5, the central processing unit CPU of the image processing unit 9 performs processing for starting a communication connection with the management center server 200 of the management center. Thereby, the first communication unit 101 of the garbage collection vehicle 100 and the second communication unit 201 of the management center server 200 are connected via a network such as a 5G mobile communication system and the Internet. At this time, information on the type of vehicle used by the garbage collection vehicle 100 and user information are transmitted to the management center server 200.

In step S6, the central processing unit CPU of the image processing unit 9 determines whether the communication connection between the first communication unit 101 and the second communication unit 201 is completed. If the connection between the first communication unit 101 and the second communication unit 201 has been completed (YES), the process advances to step S7. On the other hand, if the connection between the first communication unit 101 and the second communication unit 201 cannot be completed (NO), the process advances to step S30.

In step S7, the central processing unit CPU of the image processing unit 9 determines whether or not a reverse signal based on the driving operation of the vehicle has been input. The reverse signal is input to the image processing unit 9 via the energizing line K8 when the vehicle is operated to reverse, and the determination in step S7 is made based on the presence or absence of this reverse signal. If the reverse signal has not been input (NO), the process proceeds to step S8, and if the reverse signal has been input (YES), the process proceeds to step S23.

In step S8, the central processing unit CPU of the image processing unit 9 determines whether a loading line signal, which is a work signal, has been input. The loading line signal is input to the image processing unit 9 via the energizing line K10 when the PTO switch SWP is on and the changeover switch SW6 is switched to the loading side. Based on the presence or absence of this loading line signal, the determination in step S8 is made. If the loading line signal has been input (YES), the process proceeds to step S9, and if the loading line signal has not been input (NO), the process proceeds to step S30.

In step S9, the central processing unit CPU of the image processing unit 9 transmits to the management center server 200 that the loading line signal has been input. In step S10, the central processing unit CPU of the image processing unit 9 turns on the operating lamp 72.

In step S11, the central processing unit CPU of the image processing unit 9 transmits data of a video taken by the camera 71 during loading (during work) to the management center server 200 in real time. As will be described later, the management center server 200, which has received the video data from the image processing unit 9, performs person recognition processing (step T7) and selects a predetermined person detection area (first detection area Y11, second detection area Y11, second detection area It is determined whether a person is present in the detection area Y12).

Next, in step S12, the central processing unit CPU of the image processing unit 9 receives the determination result of the person recognition process (step T7) from the management center server 200 in real time. Specifically, as the determination results, whether or not a person detection signal is output (step T76) and whether or not a stop signal is output (step T78) are received from the management center server 200.

In step S13, the central processing unit CPU of the image processing unit 9 determines whether a person is detected in the first detection area Y11 or the second detection area Y12 based on the determination result of the person recognition process received in step S12. Determine whether or not. At this time, it is determined whether a person is detected in the first detection area Y11 or the second detection area Y12 based on the presence or absence of the output of the person detection signal received from the management center server 200 (step T76). Ru. If a person is detected in the first detection area Y11 or the second detection area Y12 (YES), the process proceeds to step S14, and if a person is not detected (NO), the process proceeds to step S21.

Next, in step S14, the central processing unit CPU of the image processing unit 9 determines whether a person has entered the second detection area Y12, which is the entry-prohibited area, based on the determination result of the person recognition process received in step S12. Determine whether or not. At this time, it is determined whether a person has entered the second detection area Y12 based on whether or not the stop signal received from the management center server 200 is output (step T78). If a person has entered the second detection area Y12 (YES), the process proceeds to step S15, and if no person has entered the area (NO), the process proceeds to step S19.

Next, the central processing unit CPU of the image processing unit 9 outputs an ON signal for the detection lamp 73 in step S15, does not output a person safety signal to the main body control unit PLC in step S16, and then proceeds to step S17. move on. Further, the central processing unit CPU outputs an on signal for the detection lamp 73 in step S19, outputs a person safety signal to the main body control unit PLC in step S20, and then proceeds to step S17. Further, the central processing unit CPU does not output an on signal for the detection lamp 73 in step S21, outputs a person safety signal to the main body control unit PLC in step S22, and then proceeds to step S17.

If a backward signal has been input in the determination in step S7 described above (YES), in step S23, the central processing unit CPU of the image processing unit 9 transmits to the management center server 200 that a backward signal has been input. . In step S24, the central processing unit CPU of the image processing unit 9 does not output an on signal for the operating lamp 72 and an off signal for the detection lamp 73, and proceeds to step S25.

In step S25, the central processing unit CPU of the image processing unit 9 transmits data of the moving image taken by the camera 71 at the time of reversing to the management center server 200 in real time. As will be described later, the management center server 200 that has received the video data from the image processing unit 9 performs person recognition processing (step T19) to detect a person in a predetermined person detection area (third detection area Y2). Determine whether it is included.

Next, in step S26, the central processing unit CPU of the image processing unit 9 receives the determination result of the person recognition process (step T19) from the management center server 200 in real time. Specifically, whether or not a person detection signal is output is received from the management center server 200.

In step S27, the central processing unit CPU of the image processing unit 9 determines whether a person is detected in the third detection area Y2 based on the determination result of the person recognition process received in step S26. At this time, it is determined whether or not a person is detected in the third detection area Y2 based on whether or not the person detection signal received from the management center server 200 is output. If a person is detected in the third detection area Y2 (YES), in step S28, the central processing unit CPU of the image processing unit 9 activates the buzzer 91 to output a backward warning sound, and then in step S28, Proceed to S17. On the other hand, if no person is detected in the third detection area Y2 (NO), the central processing unit CPU of the image processing unit 9 does not activate the buzzer 91 in step S29, and then the process proceeds to step S17.

In step S17, the central processing unit CPU of the image processing unit 9 determines whether the operator has performed an operation of switching the PTO switch SWP from on to off. If the PTO switch SWP has been turned off (YES), the process proceeds to step S18, and if the PTO switch SWP has not been turned off (NO), the process returns to step S4 and the above-described process is repeated.

Thereafter, in step S18, the central processing unit CPU of the image processing unit 9 performs processing for terminating the communication connection with the management center server 200, and ends the routine. As a result, the communication connection between the first communication unit 101 of the garbage collection vehicle 100 and the second communication unit 201 of the management center server 200 is released.

Next, the flow of control executed by the management center server 200 of the management center will be described with reference to the flowchart of FIG. As shown in FIG. 10, in step T1 after the start, when the management center server 200 receives a communication connection start signal from the image processing unit 9 of the garbage collection vehicle 100, in step T2, the second The communication unit 201 and the first communication unit 101 of the garbage collection vehicle 100 are connected via a network such as a 5G mobile communication system and the Internet. At this time, the management center server 200 receives the vehicle type information and user information transmitted from the image processing unit 9 of the garbage collection vehicle 100. The management center server 200 stores the received information on the type of garbage collection vehicle 100 used and user information in the server-side storage unit 205.

In step T3, the management center server 200 determines whether or not it has received from the image processing unit 9 of the garbage collection vehicle 100 that a vehicle backward signal has been input. The fact that the vehicle's reverse signal has been input is transmitted from the image processing unit 9 in the process of step S23 described above, and if the input of the vehicle's reverse signal is not received (NO), The process proceeds to step T4, and if it is received that a backward signal has been input (YES), the process proceeds to step T17.

In step T4, the management center server 200 determines whether or not it has received from the image processing unit 9 of the garbage collection vehicle 100 that a loading line signal has been input. The fact that the loading line signal has been input is transmitted from the image processing unit 9 in the process of step S9 described above, and when it is received that the loading line signal has been input (YES), The process proceeds to step T5, and if the input of the loading line signal is not received (NO), the process proceeds to step T15.

In step T5, the management center server 200 sets the dictionary data used for person recognition processing to the dictionary data for loading (work), and also sets the detection area used for person detection judgment to the dictionary data for loading (work). Set as a person detection area. The loading dictionary data is referred to in the person recognition process (step T7) described later, and is stored in the server-side storage unit 205 of the management center server 200. The loading dictionary data is obtained by taking images of many people's heads in advance and extracting features such as size and shape using machine learning. The features extracted as the loading dictionary data mainly include the shape of the head when the person is viewed from above. The person detection area for loading is used for person detection determination, and is stored in the server-side storage unit 205 of the management center server 200. As described above, the person detection area for loading consists of the first detection area Y11, which is set as a normal area when workers carry out garbage loading work, and the second detection area, which is a no-entry area. Y12, and each area is stored in the server-side storage unit 205 of the management center server 200.

In this embodiment, the loading dictionary data is set for each vehicle type and user of the garbage collection vehicle 100, and the loading dictionary data for each vehicle type of the garbage collection vehicle 100 and the user The loading dictionary data for each is stored in the server-side storage unit 205. Then, the loading dictionary data is set based on the information on the type of garbage collection vehicle 100 used and the user information received in step T2. That is, based on the vehicle type information and user information of the garbage collection vehicle 100 transmitted from the image processing unit 9, the loading dictionary data referred to in the determination by the image analysis unit 202 is switched.

Next, in step T6, the management center server 200 receives data of a video taken by the camera 71 during loading (during work) from the image processing unit 9 in real time. At this time, data of the moving image during loading transmitted from the image processing unit 9 in the process of step S11 described above is received.

In step T7, the management center server 200 causes the image analysis unit 202 to perform a person recognition process based on the data of the moving image during loading received in step T6. Details of the person recognition process executed by the image analysis unit 202 of the management center server 200 will be described with reference to the flowchart in FIG. 11.

First, in step T71 after the start, the image analysis unit 202 performs binarization processing on video data (input image data) captured by the camera 71 and input to the management center server 200 via the image processing unit 9. It will be done. This binarization process is, for example, a process in which when the brightness value of each pixel of input image data is equal to or greater than a preset threshold value, it is set as the maximum brightness value, and when it is less than the threshold value, it is set as the minimum brightness value. The generated binary image data has most of the effects of noise and changes in light amount removed.

Next, in step T72, labeling processing is performed by the image analysis unit 202. This labeling process is a process of regionalizing each pixel that is close to each other in the binarized image data. For example, it is a process in which multiple pixels that belong to the same luminance value and are close to each other within a predetermined distance are considered as one region. . The labeling process is performed on the entire image plane, and each region defined as one area is recognized as an object image.

Then, in steps T73 to T75, the image analysis unit 202 performs person identification processing for each object image detected within the first detection area Y11 or the second detection area Y12. In this embodiment, a camera 71 is disposed above the garbage inlet 4 to photograph the vicinity of the lower garbage inlet 4 from approximately directly above. Therefore, in the image, the head of the person H is displayed in a large size, as shown in an example in FIG. 8, and a part of the face seen from above is also displayed. Therefore, in step T73, preset loading dictionary data is referred to, and an object image that satisfies the conditions of this loading dictionary data is tentatively determined to be the head of the person H.

Next, in steps T74 and T75, the object image tentatively determined to be a head is distinguished from other object images. First, in step T74, a change in the position of the object image is detected by differential processing of a plurality of image data inputted in time series from the camera 71 (processing to find the difference in luminance value for each pixel). The moving speed of the object image is calculated by dividing this change in position, that is, the moving distance of the object image, by the time required for the change (time interval for acquiring image data).

Next, in step T75, it is determined whether the moving speed calculated in step T74 is less than or equal to a preset threshold. This threshold value is set in advance through experiments or the like so that the moving speed of the head of the worker who throws garbage into the garbage inlet 4 can be distinguished from the moving speed of the thrown garbage. If the calculated moving speed is higher than the threshold value, the movement is too fast, so it is determined that it is not the head of person H, a negative determination is made (NO), and the routine is ended (END).

On the other hand, if it is determined in step T75 that the calculated moving speed is less than or equal to the threshold (YES), it is determined that the object image is the head of person H (main determination), and the process proceeds to step T76. Outputs a detection signal.

Next, in step T77, it is determined whether the object image determined to be the head of the person H is within the second detection area Y12 described above. The determination in step T77 can be made based on whether the position coordinates of pixels forming the outline of the area forming the object image are even partially included in the second detection area Y12.

If it is affirmatively determined that the object image identified as the head of the person H is within the second detection area Y12 (YES), the process proceeds to step T78 and the operation of the garbage loading device is stopped. Outputs a signal and ends the routine (end). On the other hand, if a negative determination is made that the object image does not enter the second detection area Y12 (NO), the routine is ended without outputting a stop signal (END).

Returning to the flowchart of FIG. 10, in step T8, the management center server 200 determines that the object image determined to be the head of person H in the person recognition process of step T7 is used as an image for machine learning of the loading dictionary data. Determine whether the item belongs to a user who can collect it. If the image belongs to a user who can collect images (YES), the process proceeds to step T9, and if the image does not belong to a user who can collect images (NO), the process proceeds to step T10.

In step T9, the management center server 200 uses the learning image extracting unit 203 to extract image data of the object image at a predetermined timing from the video data acquired by the camera 71, and extracts the image data for the learning image for loading (work). It is saved in the server-side storage unit 205 as image data. The learning image data for loading (work) is used for machine learning of the dictionary data for loading, and is a video acquired by the camera 71 when there is an object image determined to be a person. The learning image extraction unit 203 extracts the data from the data at a predetermined timing.

The learning image data for loading (work) is stored in the server-side storage unit 205 in association with the position information and time information at the time of extraction. The extraction timing (acquisition timing) of learning image data for loading (work) is, for example, at the start of loading work, when the loading switch SW5 is operated, when the emergency stop switch (emergency stop buttons 60, 61, There are times when the stop plate 62) is operated, when loading work is completed, etc. Among these, the start and end of the loading work are the timings when the PTO as the drive source of the garbage loading device (work device) is turned on or off.In other words, the PTO switch SWP as the drive source switch is turned on or off. This is the timing when it is turned on or off. Moreover, when the loading switch SW5 is operated, it is the timing when the actuator operation unit that drives the work actuator that drives the garbage loading device (work device) is operated. Then, based on the learning image data for loading (work) extracted by the learning image extraction unit 203, machine learning is performed by the machine learning unit 204, and the update dictionary for loading (work) is Data is created. Furthermore, the loading (work) dictionary data stored in the server-side storage unit 205 is updated based on the loading (work) update dictionary data created by the machine learning unit 204. .

Next, in step T10, the management center server 200 determines whether a person is detected within the first detection area Y11 or the second detection area Y12. Furthermore, in step T11, the management center server 200 determines whether a person has entered the second detection area Y12, which is the entry-prohibited area.

The determinations in steps T10 and T11 are performed based on the person recognition process in step T7. Specifically, if a person detection signal is output and a stop signal is output in the person recognition process in step T7 (YES in step T10, YES in step T11), the process proceeds to step T12, and the management center server 200 , the determination result in the person recognition process in step T7 is transmitted to the image processing unit 9 of the garbage collection vehicle 100. At this time, the management center server 200 transmits a determination result indicating that a person has entered the second detection area Y12 to the image processing unit 9 of the garbage collection vehicle 100. For example, it is sufficient to transmit to the image processing unit 9 of the garbage collection vehicle 100 that a person detection signal was output in the person recognition process in step T7 and that a stop signal was output.

If the person recognition process in step T7 outputs a person detection signal but does not output a stop signal (YES in step T10, NO in step T11), the process proceeds to step T13, and the management center server 200 performs the process in step T7. The determination result in the person recognition process is transmitted to the image processing unit 9 of the garbage collection vehicle 100. At this time, the management center server 200 transmits a determination result indicating that a person has been detected within the first detection area Y11 or the second detection area Y12 to the image processing unit 9 of the garbage collection vehicle 100. For example, it is sufficient to transmit to the image processing unit 9 of the garbage collection vehicle 100 that a person detection signal was output in the person recognition process in step T7 and that a stop signal was not output.

On the other hand, if the person detection signal is not output in the person recognition process in step T7 (NO in step T10), the process proceeds to step T14, and the management center server 200 outputs the determination result in the person recognition process in step T7 to the garbage collection vehicle. 100 image processing units 9. At this time, the management center server 200 transmits a determination result indicating that no person was detected within the first detection area Y11 or the second detection area Y12 to the image processing unit 9 of the garbage collection vehicle 100. For example, information that a person detection signal was not outputted and a stop signal was not outputted in the person recognition process in step T7 may be transmitted to the image processing unit 9 of the garbage collection vehicle 100.

If it is received in step T3 that a vehicle reverse signal has been input (YES), in step T17, the management center server 200 sets the dictionary data used for the person recognition process as the reverse dictionary data. In addition, the detection area used for human detection determination is set to the third detection area Y2 for retreating. The retreat dictionary data is referred to in the person recognition process (step T19) described later, and is stored in the server-side storage unit 205 of the management center server 200. The backward dictionary data is obtained by taking images of many people's heads in advance and extracting features such as the size and shape of the heads by machine learning. The features extracted as the reversing dictionary data are mainly the head shape when the person is viewed diagonally from above, closer to the front than the loading dictionary data mentioned above. The third detection area Y2 is used for human detection determination and is stored in the server-side storage unit 205 of the management center server 200. As described above, the third detection area Y2 includes the second detection area Y12 for loading, the first detection area Y11, and the area behind them.

In this embodiment, the reversing dictionary data is set for each vehicle type and user of the garbage collection vehicle 100, and the reversing dictionary data is set for each vehicle type of the garbage collection vehicle 100 and for each user. The loading dictionary data is stored in the server-side storage unit 205. Then, the dictionary data for reversing is set based on the information on the type of vehicle used by the garbage collection vehicle 100 and the user information received in step T2. That is, based on the vehicle type information and user information of the garbage collection vehicle 100 transmitted from the image processing unit 9, the backward dictionary data referred to in the determination by the image analysis unit 202 is switched.

Next, in step T18, the management center server 200 receives data of the moving image taken by the camera 71 at the time of reversing from the image processing unit 9 in real time. At this time, data of the moving image at the time of reversing transmitted from the image processing unit 9 in the process of step S25 described above is received.

In step T19, the management center server 200 causes the image analysis unit 202 to perform person recognition processing based on the data of the moving image at the time of reversing received in step T18. The person recognition process in step T19 is substantially the same as the person recognition process in step T7 described above (see FIG. 11), and includes binarization process, labeling process, person identification process, and the like. In the person identification process, the backward dictionary data set in step T16 is used, and an object image that satisfies the conditions of the backward dictionary data is determined to be the person H. Note that the recognition of the person H is not limited to identifying the shape of the head, but may also identify the hands, feet, etc., for example.

Then, in step T20, the management center server 200 determines whether the object image determined to be the head of person H in the person recognition process in step T19 belongs to a user that can be collected as an image for machine learning of backward dictionary data. Determine whether it exists or not. If the image belongs to a user who can collect images (YES), the process proceeds to step T21, and if the image does not belong to a user who can collect images (NO), the process proceeds to step T22.

In step T21, the management center server 200 uses the learning image extracting unit 203 to extract image data of an object image at a predetermined timing from the video data acquired by the camera 71, and stores it as learning image data for reversing on the server side. It is saved in the storage unit 205. The learning image data for reversing is used for machine learning of the dictionary data for reversing, and when there is an object image determined to be a person, the learning image data for reversing is created from the video data acquired by the camera 71. It is extracted by the extraction unit 203 at a predetermined timing. The learning image data for reversing is stored in the server-side storage unit 205 in association with the position information and time information at the time of extraction. The extraction timing (acquisition timing) of the learning image data for reversing is set to be every predetermined time, such as every few seconds, for example. The machine learning unit 204 then performs machine learning based on the backward learning image data extracted by the learning image extraction unit 203, and creates backward updating dictionary data. Further, based on the update dictionary data for retreat created by the machine learning unit 204, the dictionary data for retreat stored in the server-side storage unit 205 is updated.

Next, in step T22, the management center server 200 determines whether a person is detected in the third detection area Y2. The determination in step T22 is performed based on the person recognition process in step T19. Specifically, when a person detection signal is output in the person recognition process in step T19, the process proceeds to step T23, and the management center server 200 transmits the determination result in the person recognition process in step T19 to the image processing unit of the garbage collection vehicle 100. Send to 9. At this time, the management center server 200 transmits a determination result indicating that a person has been detected in the third detection area Y2 to the image processing unit 9 of the garbage collection vehicle 100. For example, the fact that a person detection signal has been output in the person recognition process in step T19 may be transmitted to the image processing unit 9 of the garbage collection vehicle 100.

On the other hand, if a person detection signal is not output in the person recognition process in step T19, the process proceeds to step T24, and the management center server 200 transmits the determination result in the person recognition process in step T19 to the image processing unit 9 of the garbage collection vehicle 100. Send to. At this time, the management center server 200 transmits a determination result indicating that no person was detected in the third detection area Y2 to the image processing unit 9 of the garbage collection vehicle 100. For example, the fact that a person detection signal was not output in the person recognition process in step T19 may be transmitted to the image processing unit 9 of the garbage collection vehicle 100.

Then, in step T15, the management center server 200 determines whether or not a communication connection termination signal has been received from the image processing unit 9 of the garbage collection vehicle 100. If the communication connection termination signal is received from the image processing unit 9 (YES), the process proceeds to step T16, where the management center server 200 performs processing to terminate the communication connection with the image processing unit 9, and ends the routine. On the other hand, if the communication connection termination signal has not been received from the image processing unit 9 (NO), the process returns to step T3 and the above-described process is repeated.

In the person recognition system of this embodiment, by referring to the dictionary data stored in the memory M of the management center server 200, a person detection area (first detection area) near the garbage loading device of the garbage collection vehicle 100 is detected. Y11, second detection area Y12, and third detection area Y2). As a result, the management center server 200 can have the dictionary data instead of each garbage collection vehicle 100 having the dictionary data, so the management center server 200 can manage the dictionary data all at once. Therefore, the dictionary data can be updated all at once on the management center side without having the garbage collection vehicle 100 come to the service station. As a result, dictionary data can be easily updated. Further, since a high-performance machine learning function for updating dictionary data need only be provided in the management center server 200, an increase in the cost of the garbage collection vehicle 100 can be suppressed.

Further, video data acquired by the camera 71 from the garbage collection vehicle 100 is transmitted to the management center server 200 in real time, and based on the video data, the image analysis unit 202 detects a person near the garbage input port 4. It is determined whether or not there is a person in the designated area. Then, the management center server 200 transmits the determination result of the image analysis unit 202 to the garbage collection vehicle 100 in real time, and the garbage collection vehicle 100 controls the operation of the garbage loading device based on the determination result. There is. In this way, the video data acquired by the camera 71 and the judgment results of the image analysis unit 202 can be exchanged in real time between the garbage collection vehicle 100 and the management center server 200. The management center server 200 can immediately switch to and use the most suitable dictionary data according to the environment in which the garbage collection vehicle 100 is used, where the collection location and the like change. In particular, it becomes possible to switch dictionary data multiple times even during one day's collection work of the garbage collection vehicle 100.

In addition, since the garbage collection vehicle 100 receives the determination result of the image analysis unit 202 transmitted from the management center server 200 only when the garbage loading device, which is a working device, is in operation, the garbage collection vehicle 100 and the management center By avoiding unnecessary communication between the servers 200, the processing load on the management center server 200 can be reduced, and more garbage collection vehicles 100 can efficiently use the person recognition system.

In addition, the management center server 200 extracts learning image data (learning image data for loading, learning image data for reversing) from the video data acquired by the camera 71 at a predetermined timing, and stores the image data on the server side. Based on the learning image extracting unit 203 stored in the storage unit 205 and the learning image data extracted by the learning image extracting unit 203, updating dictionary data (updating dictionary data for loading, Based on the update dictionary data created by the machine learning unit 204, the dictionary data stored in the server-side storage unit 205 (update dictionary data for loading The dictionary data, backward dictionary data) are updated.

Furthermore, the management center server 200 can easily collect learning image data for updating dictionary data from many garbage collection vehicles 100 used by many users. Furthermore, when the management center server 200 extracts learning image data from video data acquired by the camera 71, the learning image extraction unit 203 extracts the learning image data at a predetermined timing. It is possible to extract learning image data in which the image, motion, background type, etc. are specified. Thereby, compared to the case where learning image data is extracted at irregular timings, it is possible to narrow down the characteristics of a person to be subjected to machine learning in advance. In addition, since the learning image data extracted at a predetermined timing is stored in the server side storage unit 205, this stored learning image data is used to store dictionary data such as the usage environment of each garbage truck 100. can be updated to suit. Thereby, it is possible to suppress erroneous recognition in the person recognition process, improve the accuracy of the person recognition process, and ensure the safety of workers and the like.

In addition, in the management center server 200, dictionary data (updated dictionary data for loading, updated dictionary data for backing up) stored in the server-side storage unit 205 is stored for each type of garbage collection vehicle 100 and for each user. The dictionary data to be referred to when making a determination by the image analysis unit 202 is switched based on the vehicle type information and user information of the garbage collection vehicle 100 transmitted from the garbage collection vehicle 100. As a result, even if there is a major change in the type of garbage truck 100 being used by a user or in the usage method of the garbage truck 100, the management center server 200 will You can immediately switch to dictionary data suitable for your environment, etc. In addition, the management center server 200 can perform person recognition processing using dictionary data suitable for the vehicle type of each garbage truck 100 or the usage environment of each user, thereby suppressing misrecognition in person recognition processing. As a result, the accuracy of person recognition processing can be improved, and the safety of workers and the like can be ensured.

-Other embodiments-
The embodiments disclosed herein are illustrative in all respects, and are not the basis for restrictive interpretation. The technical scope of the present invention is not to be interpreted only by the above-described embodiments, but is defined based on the claims. Further, the technical scope of the present invention includes all changes within the meaning and scope equivalent to the scope of the claims.

In the above embodiment, a case has been described in which the garbage collection vehicle 100 is provided with the main body control section PLC and the image processing unit 9, but the garbage collection vehicle 100 may be provided with only a single control section. For example, the main body control section PLC may be provided with the function of the image processing unit 9, and the main body control section PLC may be caused to execute the flowchart of FIG.

The above-mentioned areas for person detection (first detection area Y11, second detection area Y12, and third detection area Y2) are merely examples, and areas other than those described above may be adopted. For example, the entire area in front of the rear edge 4b of the garbage inlet 4 may be set as the second detection area Y12 (intrusion prohibited area). In this case, in the determination in step T77, if the position coordinates of the pixels forming the outline of the area forming the object image described above are included in the front side of the rear edge 4b of the garbage inlet 4, It may be determined that an object image identified as a person has been detected in the second detection area Y12.

Further, in the above embodiment, the camera 71 is used as the object image acquisition unit, but the object image acquisition unit may be other than the camera 71, for example, a distance image sensor, a stereo camera, 3D-LIDAR, etc. It may be possible to acquire images and measure distances.

In the above embodiment, a case is described in which the present invention is embodied as a garbage collection vehicle 100 equipped with a so-called rotary plate type garbage loading device, and the main parts of the garbage loading device are formed by the rotating plate 10 and the pushing plate 20. It is configured. However, the main part of the garbage loading device is not limited to this, and the main part of the garbage loading device may be constituted by a lifting plate and a pushing plate, and the structure is not particularly limited. Furthermore, the present invention can be applied to a person recognition system for specially equipped vehicles other than garbage collection vehicles, such as dump trucks, mixer trucks, tank trucks, container handling vehicles, and suction trucks, and can be used as a work device mounted on a vehicle chassis. Various types of equipment other than garbage loading equipment can be considered.

INDUSTRIAL APPLICABILITY The present invention can be used in a person recognition system for a specially equipped vehicle with a working device mounted on the chassis, and can be used, for example, in a garbage loading device disposed inside a garbage disposal box, and in a garbage loading device attached to the garbage disposal box. The present invention can be used in a person recognition system for a garbage collection vehicle, which is equipped with an object image acquisition unit that acquires an image of the vicinity of the garbage input port of the input box.

1 Vehicle chassis 3 Garbage input box 4 Garbage input port 9 Image processing unit 71 Camera (object image acquisition unit)
100 Garbage collection vehicle (specially equipped vehicle)
200 Management center server 202 Image analysis section 205 Server side storage section

Claims (4)

A person recognition system for a specially equipped vehicle with a working device mounted on the chassis,
The specially equipped vehicle is
an object image acquisition unit capable of acquiring a moving image of an object near the work device;
Equipped with a control unit that can communicate with the management center server,
The management center server is
a storage unit that stores video data acquired by the object image acquisition unit and transmitted from the specially equipped vehicle;
An image for determining whether or not a person is in a person detection area near the working device by referring to dictionary data stored in the storage unit based on video data transmitted from the specially equipped vehicle. Equipped with an analysis department,
The control unit controls the operation of the work device based on the determination result of the image analysis unit transmitted from the management center server ,
The dictionary data is provided separately for each type of specially equipped vehicle or for each user,
The management center server switches the dictionary data referred to in the determination by the image analysis unit based on the vehicle type information and user information of the specially equipped vehicle transmitted from the specially equipped vehicle. . The person recognition system for specially equipped vehicles according to claim 1,
The management center server is
a learning image extraction unit that extracts learning image data for updating the dictionary data at a predetermined timing from video data transmitted from the specially equipped vehicle;
a machine learning unit that creates update dictionary data by machine learning based on the learning image data extracted by the learning image extraction unit;
A person recognition system for a specially equipped vehicle, wherein the dictionary data stored in the storage section is updated based on update dictionary data created by the machine learning section. The person recognition system for a specially equipped vehicle according to claim 1 or 2,
A person recognition system for a specially equipped vehicle, wherein the control unit receives the determination result of the image analysis unit transmitted from the management center server only while the work device is in operation. The person recognition system for a specially equipped vehicle according to any one of claims 1 to 3 ,
The working device is a garbage loading device disposed inside a garbage input box,
The image analysis unit is configured to determine whether or not a person is in a person detection area near a garbage input port opened at the back of the garbage input box, based on video data transmitted from the specially equipped vehicle. A person recognition system for specially equipped vehicles.

Images