JP6834372B2 - Information processing equipment, information processing systems, information processing methods and programs - Google Patents

Description

The present invention relates to information processing devices, information processing systems, information processing methods and programs.

At product production sites, the issue is how to quantitatively grasp the movements of workers such as walking and movement in the process involving humans, and how to quickly analyze and improve them. As a technology for quantifying human movements, efforts to "visualize" human movements as "traffic lines" are widespread in the world. Technologies for acquiring motion lines include those that use the location information of a wireless LAN (Local Area Network), those that attach an ultrasonic transmitter to the tracking target and specify the position with a receiver, and beacons. There are technologies such as those that capture the position by receiving the ID that transmits However, in order to carry out tracking with these technologies, it is necessary to give the tracked object a wireless LAN, a terminal such as an ultrasonic transmission, or an ID. In addition, as a method of acquiring a flow line without having a terminal as a tracking target, a technique of tracking a tracking target shown in an image taken by a camera with an image processing technique and specifying a position is already known. ing. After acquiring the coordinates of the movement line, the movement line is superimposed on the image according to the coordinates, and the repetitive work (standard work) of which work process and how long the work was done is analyzed. Improvements can be made.

As a technique for drawing a flow line and analyzing the work of an operator, the tracking target is divided into a frame reading means for reading a frame of a video captured by the tracking target and a frame read by the frame reading means. Based on the tracking target area setting means for setting the tracking target area for, the tracking means for tracking the tracking target area among a plurality of frames read by the frame reading means, and the tracking result of the tracking means, the operation trajectory of the tracking target is determined. A technique of a locus drawing device including a drawing means for drawing and a display control means for displaying an operation locus drawn by the drawing means on a display device is disclosed (see Patent Document 1).

However, in the technique described in Patent Document 1, the flow line is once drawn as to which part of the flow line indicating a series of repetitive work (standard work) corresponds to the work area where the specific work is performed. Therefore, there was a problem that the work area could only be set manually while looking at the flow line.

The present invention has been made in view of the above, and an object of the present invention is to provide an information processing device, an information processing system, an information processing method, and a program capable of determining a work area from an acquired flow line.

In order to solve the above-mentioned problems and achieve the object, the present invention includes a first acquisition unit that acquires a plurality of tracking result information including position information of a tracking target in a routine work which is a series of operations of a predetermined unit. With the first extraction unit that extracts the stagnant portion, which is a group of stagnant points, based on the moving distance of the points indicated by the position information of the tracking target of the plurality of tracking result information acquired by the first acquisition unit. , Based on an enlarged portion that obtains an enlarged region obtained by expanding a predetermined point based on the stagnant portion corresponding to each tracking result information, and an region in which at least one of a plurality of the enlarged regions obtained by the enlarged portion overlaps. It is characterized by having a first determination unit for determining a work area.

According to the present invention, the work area can be determined from the acquired flow line.

FIG. 1 is a diagram showing an example of the overall configuration of the image processing system according to the embodiment. FIG. 2 is a diagram showing an example of the hardware configuration of the information processing apparatus according to the embodiment. FIG. 3 is a diagram showing an example of the configuration of the functional block of the information processing apparatus according to the embodiment. FIG. 4 is a flowchart showing an example of the flow of the color registration process of the information processing apparatus according to the embodiment. FIG. 5 is a diagram showing an example of a state in which an image is displayed on a tracking screen displayed on the display unit of the information processing apparatus according to the embodiment. FIG. 6 is a diagram illustrating an operation of registering a color to be tracked in the information processing apparatus according to the embodiment. FIG. 7 is a diagram illustrating a tracking process for tracking a tracking target in the information processing apparatus according to the embodiment. FIG. 8 is a diagram showing an example of tracking result data showing information of a tracking target detected in the tracking process of the information processing apparatus according to the embodiment. FIG. 9 is a flowchart showing an example of a flow of determination processing of a standard work area of the information processing apparatus according to the embodiment. FIG. 10 is a diagram showing an example of a plurality of flow lines of routine work used for obtaining a fixed work area in the information processing apparatus according to the embodiment. FIG. 11 is a diagram showing an example of a state in which a plurality of flow lines of the standard work used for obtaining the standard work area are thickened. FIG. 12 is a diagram showing an example in which the extracted regions are continuous. FIG. 13 is a diagram showing an example in which the extracted regions are continuous. FIG. 14 is a diagram illustrating an operation of determining a routine work area in the information processing apparatus according to the embodiment. FIG. 15 is a flowchart showing an example of a flow of determination processing of a work area of the information processing apparatus according to the embodiment. FIG. 16 is a diagram showing an example of a stagnant area and a sudden action area. FIG. 17 is a diagram illustrating an operation of determining a work area in the information processing apparatus according to the embodiment. FIG. 18 is a diagram showing an example of a work area and a moving area determined in the information processing apparatus according to the embodiment.

Hereinafter, embodiments of the information processing apparatus, information processing system, information processing method, and program according to the present invention will be described in detail with reference to the drawings. Further, the present invention is not limited to the following embodiments, and the components in the following embodiments include those easily conceived by those skilled in the art, substantially the same, and so-called equal ranges. Includes. Furthermore, various omissions, substitutions, changes and combinations of components can be made without departing from the gist of the following embodiments.

(Overall configuration of image processing system)
FIG. 1 is a diagram showing an example of the overall configuration of the image processing system according to the embodiment. The overall configuration of the image processing system 1 of the present embodiment will be described with reference to FIG.

As shown in FIG. 1, the image processing system 1 of the present embodiment includes image pickup devices 2a to 2d, an information processing device 3, a hub 4, and an external device 10.

The image pickup devices 2a to 2d capture (shoot) the subject by converting light from the subject into an electric signal, and move images (for example, 10 [FPS], 25 [FPS]] composed of a plurality of frames (image data). Etc.), which is a video camera that generates video data. For example, the image pickup devices 2a to 2d take an image of a worker working in a production facility or a production line that produces a product, and generate video data.

When the imaging devices 2a to 2d are referred to without distinction, or when they are collectively referred to, they are simply referred to as "imaging device 2". Further, in FIG. 1, the image processing system 1 has a configuration including four image pickup devices 2, but the present invention is not limited to this, and the image processing system 1 may include another number of image pickup devices 2. Good.

The information processing device 3 is a PC (Personal Computer), a workstation, or the like that functions as an image processing device that executes image processing based on the video data captured by each image pickup device 2. The information processing device 3 transmits video data to be image-processed and image-processed result information (position information, etc.) to the external device 10 via a communication cable or a network. Further, the information processing device 3 receives the video data captured in the past and the result information of the image processing of the video data from the external device 10 via the communication cable or the network. Therefore, the information processing device 3 can communicate with the external device 10 via the communication cable or the network.

The hub 4 is a line concentrator corresponding to the Ethernet (registered trademark) standard for connecting the image pickup devices 2a to 2d to the information processing device 3. When the hub 4 is compatible with the Ethernet standard, for example, data communication is performed between the image pickup devices 2a and 2d and the information processing device 3 by a protocol such as TCP (Transmission Control Protocol) / IP (Internet Protocol). In this case, the image pickup devices 2a to 2d and the information processing device 3 have a MAC (Media Access Control) address for communicating by the TCP / IP protocol, and an IP address such as a private IP address is assigned.

The hub 4 shown in FIG. 1 shows an example of relaying communication by the TCP / IP protocol, but the hub 4 is not limited to this. For example, the information processing device 3 has a VGA (Video Graphics Array) terminal or a USB (Universal Serial Bus) port, and a plurality of image pickup devices 2 are concentrated on a hub 4 via a VGA cable or a USB cable to be an information processing device. It may be in the form of being connected to 3.

Further, although the configuration in which each imaging device 2 is connected to the information processing device 3 via the hub 4 has been described, the present invention is not limited to this, and a network such as a LAN (Local Area Network), a dedicated line, or the Internet is used. It may be configured to communicate with the information processing device 3 via.

The external device 10 is an external storage device, a database server, or the like that stores image data captured by each image pickup device 2 and image processing result information such as position information indicating the position of a person (worker) in the image data. Is.

As described above, the number of image pickup devices 2 for collecting video data is not limited, but in the following description, as shown in FIG. 1, a configuration in which four image pickup devices 2 are connected will be described as an example. To do.

(Hardware configuration of information processing device)
FIG. 2 is a diagram showing an example of a hardware configuration of the information processing apparatus according to the embodiment. An example of the hardware configuration of the information processing apparatus 3 of the present embodiment will be described with reference to FIG.

As shown in FIG. 2, the information processing device 3 includes a CPU (Central Processing Unit) 101, a ROM (Read Only Memory) 102, a RAM (Random Access Memory) 103, an external storage device 104, a display 105, and the like. It includes a network I / F 106, a keyboard 107, a mouse 108, a DVD (Digital Peripheral Disc) drive 109, an external device I / F 111, and a speaker 112.

The CPU 101 is a device that controls the operation of the entire information processing device 3. The ROM 102 is a non-volatile storage device that stores programs such as a BIOS (Basic Input / Output System) for the information processing device 3 and firmware. The RAM 103 is a volatile storage device used as a work area of the CPU 101.

The external storage device 104 is an HDD (Hard Disk Drive) or SSD (Solid) that stores various data such as video data received from the image pickup device 2 and the external device 10, image processing result information (position information, etc.), and setting information. It is a storage device such as State Drive).

The display 105 is a display device that displays various information such as cursors, menus, windows, characters or images, or a screen of an application that executes image processing by the information processing device 3. The display 105 is, for example, a CRT (Cathode Ray Tube) display, an LCD (Liquid Crystal Display) display, an organic EL (Electroluminance) display, or the like. The display 105 is connected to the main body of the information processing device 3, for example, by a VGA cable, an HDMI (registered trademark) (High-Definition Multimedia Interface) cable, or the like, or is connected by an Ethernet cable.

The network I / F 106 is an interface for connecting to the hub 4 for data communication. The network I / F 106 is, for example, a NIC (Network Interface Card) that enables communication using the TCP / IP protocol. Specifically, the information processing device 3 receives video data from the image pickup device 2 via the network I / F 106.

The keyboard 107 is an input device for selecting characters, numbers, various instructions, moving a cursor, setting setting information, and the like. The mouse 108 is an input device for selecting and executing various instructions, selecting a processing target, moving a cursor, setting setting information, and the like.

The DVD drive 109 is a device that controls reading, writing, and deleting of data to a DVD 110 as an example of a removable storage medium.

The external device I / F 111 is an interface for connecting to an external device (for example, the external device 10) and performing data communication. The external device I / F 111 is, for example, a NIC or a USB interface card that enables communication using the TCP / IP protocol. Specifically, the information processing device 3 performs data communication with the external device 10 via the external device I / F 111.

The speaker 112 is a voice output device that notifies by voice when an abnormality is determined in an abnormality determination for working time or the like, which will be described later.

The CPU 101, ROM 102, RAM 103, external storage device 104, display 105, network I / F 106, keyboard 107, mouse 108, DVD drive 109, external device I / F 111, and speaker 112 are buses such as an address bus and a data bus. They are communicatively connected to each other by 113. When the display 105 is connected by an Ethernet cable, it is connected to the network I / F 106, and in this case, data communication is performed by a protocol such as TCP / IP.

(Block configuration of image processing system)
FIG. 3 is a diagram showing an example of the configuration of the functional block of the information processing apparatus according to the embodiment. The block configuration of the information processing apparatus 3 will be described with reference to FIG. In FIG. 3, for simplification of the description, the hub 4 will be omitted from the illustration.

As shown in FIG. 3, the information processing apparatus 3 includes a video receiving unit 301, an input unit 302, a setting unit 303, a storage unit 304, a playback unit 305, a first acquisition unit 306, and a designation unit 307. , Registration unit 308, detection unit 309, locus output unit 310 (movement line generation unit), data communication unit 321, thickening unit 322, standard extraction unit 323 (second extraction unit), and continuous determination unit. 324 (first determination unit), standard determination unit 325 (second determination unit), stagnation extraction unit 326 (first extraction unit), representative point identification unit 327 (specification unit), expansion unit 328, and process. Extraction unit 329, process determination unit 330 (first determination unit), movement determination unit 331 (third determination unit), second acquisition unit 341, area determination unit 342 (second determination unit), and measurement unit. It has 343, a time determination unit 344 (third determination unit), a notification unit 345, a management unit 351 (change unit, generation unit), a display control unit 352, and a display unit 353.

The video receiving unit 301 is a functional unit that receives video data from the imaging device 2 via the hub 4. The video receiving unit 301 stores the received video data in the storage unit 304, or transmits the received video data to the external device 10 via the data communication unit 321. The video receiving unit 301 is realized by the network I / F 106 shown in FIG.

The input unit 302 is a device that performs operation input for causing the information processing device 3 to execute a color registration process, a tracking target tracking process, a routine work area and a work area determination process, which will be described later. The input unit 302 is realized by the keyboard 107 and the mouse 108 shown in FIG.

The setting unit 303 is a functional unit that sets various setting information according to an operation signal from the input unit 302 operated by the user and stores it in the storage unit 304. For example, the setting unit 303 sets information (hereinafter, may be referred to as “tracking parameter”) indicating a predetermined range centered on the reference color registered by the registration unit 308 as described later, and sets the set tracking parameter. , Stored in the storage unit 304. Here, in the predetermined range centered on the reference color, when the reference color is represented in the RGB color space, an upper limit value obtained by adding a predetermined amount to the R value of the reference color and a lower limit value obtained by subtracting the predetermined amount are set. As a result, a predetermined range centered on the R value of the reference color is determined. The same applies to the G value and B value of the reference color. The predetermined amount may be set to a separate value for each of the R value, the G value, and the B value. The setting unit 303 is realized by a program executed by the CPU 101 shown in FIG.

The storage unit 304 is a functional unit that stores various data such as video data received by the video receiving unit 301, setting information, and position information obtained by tracking processing described later. The storage unit 304 identifies and stores which image pickup device 2 generated the video data received by the video reception unit 301. The storage unit 304 is realized by, for example, the RAM 103 shown in FIG. 2 or the external storage device 104.

The playback unit 305 sends the video data received by the video reception unit 301, received by the data communication unit 321, or acquired from the storage unit 304 to the display control unit 352 according to the operation signal from the input unit 302 operated by the user. This is a functional unit that causes the feed and display control unit 352 to reproduce and display video data on the display unit 353. Specifically, for example, as shown in FIG. 5 described later, the reproduction unit 305 causes the individual display unit 401 of the tracking screen 400 displayed on the display unit 353 to reproduce and display the video data. The reproduction unit 305 is realized by a program executed by the CPU 101 shown in FIG.

The first acquisition unit 306 is a functional unit that acquires a frame from the video data received by the video reception unit 301, received by the data communication unit 321 or stored in the storage unit 304. The first acquisition unit 306 is realized by a program executed by the CPU 101 shown in FIG.

The designation unit 307 tracks the tracking target in the display area (for example, the individual display unit 401 shown in FIG. 5 to be described later) for displaying the video data in the display unit 353 according to the operation signal from the input unit 302 operated by the user. It is a functional unit that designates a designated area (for example, a designated area 550 shown in FIG. 6 to be described later) for specifying a color as feature information used in the process. The designation unit 307 stores the information of the designated designated area in the storage unit 304 in association with each image pickup device 2. The designation unit 307 is realized by a program executed by the CPU 101 shown in FIG.

The registration unit 308 is a functional unit that specifies a reference color based on a pixel value indicating a color of a designated area designated by the designation unit 307 and stores (registers) the information of the specified reference color in the storage unit 304. .. Here, the pixel value of the frame of the video data displayed on the individual display unit 401 is information indicating a color based on, for example, an RGB color space or an HSV color space. Specifically, the registration unit 308 calculates, for example, the average value of the pixel values of the pixels included in the designated area, and specifies the average value as a reference color. The registration unit 308 may specify the pixel value of the pixel in the center of the designated area as a reference color. The registration unit 308 is realized by a program executed by the CPU 101 shown in FIG.

The detection unit 309 is a functional unit that detects a tracking target based on a reference color (tracking parameter set by the setting unit 303) registered by the registration unit 308 in a frame to be tracked by the tracking target. .. Specifically, the detection unit 309 executes processing such as extraction processing, binarization processing, and labeling processing on the frame to be tracked to be tracked, and a predetermined range centered on the above-mentioned reference color. The area corresponding to the block containing the pixels of the color (hereinafter, may be referred to as “reference color range”) is detected as the tracking target. The detection unit 309 is realized by a program executed by the CPU 101 shown in FIG.

Note that the detection unit 309 detects a region corresponding to a block including pixels of a color within the reference color range as a tracking target, but the detection unit 309 is not limited to this. For example, the registration unit 308 specifies a shape existing in the designated area designated by the designation unit 307, stores (registers) information on the specified shape (an example of feature information) in the storage unit 304, and detects (registers) the detection unit 309. May detect a shape that is the same as or similar to the registered shape as a tracking target.

Further, the detection unit 309 is not limited to the operation of detecting the tracking target for each frame of the video data, and for example, the detection unit 309 detects the tracking target in the frame acquired by the first acquisition unit 306 at predetermined time intervals. May be. Further, when the tracking target is detected for each frame, the frame rate may be set, and when the tracking target is detected for each predetermined time, the predetermined time may be set.

The locus output unit 310 generates a locus line (traffic line) indicating the locus of the tracking target, and causes the display control unit 352 to superimpose the locus line (traffic line) on the video data being reproduced and displayed. Is. The locus output unit 310 is realized by a program executed by the CPU 101 shown in FIG.

The data communication unit 321 is a functional unit that transmits / receives tracking result information (described later) including video data and position information to / from the external device 10. When the data communication unit 321 receives the video data, the tracking result information, and the like, the data communication unit 321 may store these information in the storage unit 304. The data communication unit 321 is realized by the network I / F 106 shown in FIG.

The thickening unit 322 is a functional unit that thickens the flow line displayed on the display unit 353 in the determination process of the standard work area described later. The thickened portion 322 is realized by a program executed by the CPU 101 shown in FIG.

The standard extraction unit 323 is a functional unit that extracts a region in which a predetermined number or more of flow lines thickened by the thick line section 322 overlap. The standard extraction unit 323 is realized by a program executed by the CPU 101 shown in FIG.

The continuous determination unit 324 is a functional unit that determines whether or not the regions extracted by the standard extraction unit 323 are continuous. The continuous determination unit 324 is realized by a program executed by the CPU 101 shown in FIG.

The standard determination unit 325 is a functional unit that determines a region determined to be continuous by the continuous determination unit 324 as a standard work area. The fixed form determination unit 325 is realized by a program executed by the CPU 101 shown in FIG.

The stagnation extraction unit 326 is a point group indicating each coordinate included in the flow line, and is a functional unit that extracts a point group (stagnation portion) in which the tracking target is determined to be stagnant. The stagnation extraction unit 326 is realized by a program executed by the CPU 101 shown in FIG.

The representative point identification unit 327 has a function of identifying a representative point (representative point) from the points constituting the stagnation portion extracted by the stagnation extraction unit 326 in each of a plurality of flow lines to be processed for determining the standard work area. It is a department. The representative point specifying unit 327 is realized by a program executed by the CPU 101 shown in FIG.

The enlargement unit 328 is a functional unit that enlarges the representative point specified by the representative point identification unit 327. The enlargement unit 328 is realized by a program executed by the CPU 101 shown in FIG.

The process extraction unit 329 is a functional unit that extracts a region in which representative points enlarged by the enlargement unit 328 overlap by a predetermined number or more. The process extraction unit 329 is realized by a program executed by the CPU 101 shown in FIG.

The process determination unit 330 is a functional unit that surrounds the area extracted by the process extraction unit 329 with a rectangle and determines the area (work area) of the process in which the worker stagnates in the routine work. The process determination unit 330 is realized by a program executed by the CPU 101 shown in FIG.

The movement determination unit 331 is a functional unit that determines as a movement area an area excluding the area that overlaps with the work area determined by the process determination unit 330 from the standard work area determined by the standard determination unit 325. The movement determination unit 331 is realized by a program executed by the CPU 101 shown in FIG.

The second acquisition unit 341 is a functional unit that acquires a frame from the video data received by the video reception unit 301. That is, the second acquisition unit 341 acquires a frame of real-time video data. However, the frame of the video data acquired by the second acquisition unit 341 is not a real-time frame, but may be a frame of the past video data stored in the storage unit 304 or the external device 10. The second acquisition unit 341 is realized by a program executed by the CPU 101 shown in FIG.

The area determination unit 342 is a functional unit that determines whether or not a tracking target such as an operator exists in the work area or the moving area. The area determination unit 342 is realized by a program executed by the CPU 101 shown in FIG.

The measurement unit 343 is a functional unit that measures the work time or the movement time, respectively, when it is determined by the area determination unit 342 that the tracking target of the worker or the like exists in the work area or the movement area. In addition, the measurement unit 343 calculates the maximum value, the minimum value, and the average value of the working time in which the tracking target exists in the working area at a predetermined time. By calculating the maximum value, the minimum value, and the average value of the working time for a specific worker in this way, it is possible to analyze the work unevenness, efficiency, work ability, and the like of the worker. .. The measurement unit 343 may measure the time when the tracking target deviates from the moving area. The measurement unit 343 is realized by a program executed by the CPU 101 shown in FIG.

The time determination unit 344 is a function unit that determines whether or not the work time measured by the measurement unit 343 is within a predetermined range. When the measurement unit 343 measures the time when the tracking target is out of the moving area, the time determination unit 344 may determine whether or not the time is equal to or longer than a predetermined time. Further, the above-mentioned predetermined range may be a range set by the setting unit 303, or may be a range based on the time obtained by the number of points constituting the stagnation portion extracted by the above-mentioned stagnation extraction unit 326. The time determination unit 344 is realized by a program executed by the CPU 101 shown in FIG.

The notification unit 345 is a functional unit that notifies that the work time is abnormal when the time determination unit 344 determines that the work time is out of the predetermined range. As a result, there is some problem with the work in the work area of a specific worker, and it can be specified as the work to be improved. When the time determination unit 344 determines whether or not the time when the tracking target deviates from the moving area is the predetermined time or more, the notification unit 345 notifies that fact when it is determined that the time is longer than the predetermined time. May be. As a result, it is possible to recognize that some unexpected action has occurred during the movement, which is not the process of performing the work. The notification unit 345 is realized by the speaker 112 shown in FIG. The notification unit 345 is not limited to being realized by the speaker 112. For example, the notification unit 345 displays the above notification content on the display unit 353, or transmits the above notification content to a predetermined destination by e-mail. It may be realized by a function or the like.

The management unit 351 tracks the tracking target specified by the reference color registered by the registration unit 308 for the video data displayed on the tracking screen 400 according to the operation signal from the input unit 302 operated by the user, for example. It is a functional part that starts or stops processing. In addition, the management unit 351 manages the entire operation of the information processing device 3 (for example, storing various information in the storage unit 304). The management unit 351 is realized by a program executed by the CPU 101 shown in FIG.

The display control unit 352 is a functional unit that controls the display operation of the display unit 353. Specifically, the display control unit 352 causes the display unit 353 to display the video data, the setting information set by the setting unit 303, the designated area designated by the designation unit 307, and the like. The display control unit 352 is realized by a program (driver) executed by the CPU 101 shown in FIG.

The display unit 353 is a display device that displays various data under the control of the display control unit 352. The display unit 353 displays, for example, the tracking screen 400, which will be described later, by the function of the program (application) executed by the CPU 101 shown in FIG. The display unit 353 is realized by the display 105 shown in FIG.

The setting unit 303, the reproduction unit 305, the first acquisition unit 306, the designation unit 307, the registration unit 308, the detection unit 309, the locus output unit 310, the data communication unit 321 and the thick line of the information processing device 3 shown in FIG. Unit 322, standard extraction unit 323, continuous determination unit 324, standard determination unit 325, stagnation extraction unit 326, representative point identification unit 327, expansion unit 328, process extraction unit 329, process determination unit 330, movement determination unit 331, second A part or all of the acquisition unit 341, the area determination unit 342, the measurement unit 343, the time determination unit 344, the management unit 351 and the display control unit 352 are not programs that are software, but are ASICs (Application Specific Integrated Circuits) or FPGAs. It may be realized by a hardware circuit such as (Field-Programmable Gate Array).

Further, each functional unit shown in FIG. 3 conceptually shows a function, and is not limited to such a configuration. For example, a plurality of functional units shown as independent functional units in FIG. 3 may be configured as one functional unit. On the other hand, the function possessed by one functional unit in FIG. 3 may be divided into a plurality of functions and configured as a plurality of functional units.

Further, for the sake of simplification of the following description, the expression that the display control unit 352 displays the received data on the display unit 353 is simply expressed by the functional unit that sends the data to the display control unit 352. (Or, it is expressed that it is displayed on the screen displayed by the display unit 353). For example, when the reproduction unit 305 sends the video data to the display control unit 352 and causes the display control unit 352 to reproduce and display the video data on the display unit 353, the reproduction unit 305 simply sends the video data to the display unit 353. It shall be expressed as being reproduced and displayed.

(Color registration process)
FIG. 4 is a flowchart showing an example of the flow of the color registration process of the information processing apparatus according to the embodiment. FIG. 5 is a diagram showing an example of a state in which an image is displayed on a tracking screen displayed on the display unit of the information processing apparatus according to the embodiment. FIG. 6 is a diagram illustrating an operation of registering a color to be tracked in the information processing apparatus according to the embodiment. The flow of the color registration process of the information processing apparatus 3 according to the present embodiment will be described with reference to FIGS. 4 to 6.

First, in explaining the flow of the color registration process, the configuration of the tracking screen 400 will be described with reference to FIG. The tracking screen 400 is a screen for registering a color which is characteristic information by a color registration process and performing a tracking process for a specific tracking target in a video reproduced and displayed on the individual display unit 401. As shown in FIG. 5, the tracking screen 400 includes an individual display unit 401, a camera selection tab 401a, a video operation button unit 403, a seek bar 404, a slider 405, an enlargement / reduction button unit 406, and a span change button unit. It includes a 407, a time display unit 408, a registration button 409, a tracking start button 410, and a tracking stop button 411.

The individual display unit 401 receives video data received by the playback unit 305 by the video reception unit 301, received by the data communication unit 321, or acquired from the storage unit 304 according to the operation signal from the input unit 302 operated by the user. Is a display unit that is played back and displayed. In the example of FIG. 5, when the camera selection tab 401a of "2" is selected by the operation of the input unit 302 by the user, the reproduction unit 305 acquires the video data captured by the image pickup device 2b from the storage unit 304. Then, it is reproduced and displayed on the individual display unit 401.

The camera selection tab 401a is an operation unit for selecting which of the video data of the imaging devices 2a to 2d that the user wants to be reproduced and displayed to be displayed on the individual display unit 401. For example, the video data captured by the image pickup devices 2a to 2d are associated with "1" to "4", respectively, and in the example of FIG. 5, "2", that is, the video data of the image pickup device 2b is selected. An example is shown.

The video operation button unit 403 is a group of buttons for playing, rewinding, fast-forwarding, stopping, and pausing video data on the individual display unit 401 by operating the input unit 302 by the user. The seek bar 404 is a rod-shaped body that indicates the time of the video data reproduced and displayed on the individual display unit 401 depending on the position of the slider 405 arranged on the seek bar 404. The slider 405 is a shape and an operation unit that slides to a position on the seek bar 404 corresponding to the imaging time (time when the displayed frame is captured) of the video data reproduced and displayed on the individual display unit 401. On the contrary, when the slider 405 is slid by the operation of the input unit 302 by the user, the playback unit 305 displays the frame of the video data of the imaging time corresponding to the position on the seek bar 404 where the slider 405 exists as the individual display unit 401. Is displayed in.

The enlargement / reduction button unit 406 is a group of buttons in which the video data reproduced and displayed on the individual display unit 401 is enlarged or reduced by the reproduction unit 305 by the operation of the input unit 302 by the user. For example, each time the enlargement button is pressed in the enlargement / reduction button unit 406, the video data is enlarged and displayed at a predetermined enlargement ratio. For example, assuming that the normal display state is 100%, the display is enlarged as 120%, 140%, ....

The span change button unit 407 is a group of buttons whose display span of the seek bar 404 is enlarged or reduced by the reproduction unit 305 by the operation of the input unit 302 by the user. It is not necessary for the reproduction unit 305 to enlarge or reduce the display span of the seek bar 404, and different functional units may be responsible for it.

The time display unit 408 is a display area for displaying the imaging date and the imaging time of the video data reproduced and displayed on the individual display unit 401 by the reproduction unit 305.

The registration button 409 is a button that specifies the reference color in the registration unit 308 for the designated area designated by the designation unit 307 and registers (stores) the information of the specified reference color in the storage unit 304.

When the tracking start button 410 is pressed by the operation of the input unit 302 by the user, the management unit 351 performs a tracking process of the tracking target specified by the reference color in the video data reproduced and displayed on the individual display unit 401. It is a button to start. When the tracking stop button 411 is pressed by the operation of the input unit 302 by the user, the tracking target tracking process performed on the video data reproduced and displayed on the individual display unit 401 by the management unit 351 is performed. It is a button to stop.

Next, the specific flow of the color registration process will be described. The tracking screen 400 shown in FIG. 5 shows a state in which the video data corresponding to the image pickup apparatus 2b is reproduced and displayed by the reproduction unit 305.

<Step S11>
First, at the timing when the part of the image desired to be color-registered is displayed during the reproduction display of the video data on the individual display unit 401, the pause button of the video operation button unit 403 is released by the operation of the input unit 302 by the user. When pressed, the reproduction unit 305 pauses the video data reproduced and displayed on the individual display unit 401. As shown in FIG. 5, the worker 500 is shown in the paused video data frame. Then, the first acquisition unit 306 acquires the frame displayed on the individual display unit 401 from the paused video data. Then, the process proceeds to step S12.

<Step S12>
Here, it is assumed that the tracking target in the frame in which the information processing device 3 according to the present embodiment is displayed on the individual display unit 401 is the tracking target 501 of the worker 500 shown in FIG. The tracking target 501 is a part of the outerwear worn by the worker 500 and shall have a specific color.

The designation unit 307 is a frame acquired by the first acquisition unit 306 according to an operation of the input unit 302 by the user (for example, a drag operation of the mouse 108), that is, a frame displayed on the individual display unit 401. As shown in, in order to specify the color as the feature information used in the tracking process of the tracking target, the designated area 550 is designated in the tracking target 501.

It should be noted that the designated area can be designated by the designation unit 307 even if the frame displayed on the individual display unit 401 can be enlarged or reduced by the enlargement button or the reduction button of the enlargement / reduction button unit 406. Good. Then, the process proceeds to step S13.

<Step S13>
The registration unit 308 identifies the reference color based on the designated area 550 designated by the designation unit 307 by pressing the registration button 409 by the operation of the input unit 302 by the user, and stores the information of the specified reference color in the storage unit 304. Memorize (register) in. Specifically, the registration unit 308 calculates, for example, the average value of the pixel values of the pixels included in the designated area 550, and specifies the average value as the reference color.

Then, the setting unit 303 sets information indicating a predetermined range centered on the reference color registered by the registration unit 308 as a tracking parameter, and stores the set tracking parameter in the storage unit 304. By setting this tracking parameter, the reference color range is determined.

By the above steps S11 to S13, the color registration process (identification and registration of the reference color) and the setting of the tracking parameters are executed. The flow of the color registration process shown in FIG. 4 described above is an example, and the reference color may be specified and the tracking parameters may be set by other methods.

(Overview of tracking process for tracking target)
FIG. 7 is a diagram illustrating a tracking process for tracking a tracking target in the information processing apparatus according to the embodiment. FIG. 8 is a diagram showing an example of tracking result data showing information of a tracking target detected in the tracking process of the information processing apparatus according to the embodiment. The outline of the tracking process of the tracking target by the information processing apparatus 3 according to the present embodiment will be described with reference to FIGS. 7 and 8. It is assumed that the reference color is registered in advance by the above-mentioned color registration process. Then, during the playback display of the video data by the playback unit 305 on the individual display unit 401, the tracking start button 410 is pressed by the operation of the input unit 302 by the user at the timing when the frame desired to be tracked for the tracking target is displayed. When pressed, the tracking process of the tracking target is started by the management unit 351 from the frame displayed by the reproduction unit 305. The video data reproduced and displayed on the individual display unit 401 by the reproduction unit 305 may be real-time video data received from the image pickup device 2 via the image reception unit 301, or is stored in the storage unit 304. Alternatively, it may be past video data received from the external device 10 via the data communication unit 321.

The first acquisition unit 306 acquires a frame of video data reproduced and displayed by the reproduction unit 305. The second and subsequent frames acquired by the first acquisition unit 306 are not limited to the frames following the frame for which the tracking process has been completed, and the corresponding frames may be acquired after a predetermined time. .. That is, the tracking process is not limited to being executed for each frame after the frame of the video data from which the process is started, and may be executed for each frame at a predetermined time.

Next, the detection unit 309 detects the tracking target in the frame acquired by the first acquisition unit 306 based on the reference color (tracking parameter set by the setting unit 303) registered by the registration unit 308. Specifically, the detection unit 309 executes processing such as extraction processing, binarization processing, and labeling processing on the frame, and includes pixels of colors that fall within the reference color range centered on the above-mentioned reference color. The area corresponding to the block is detected as a tracking target (tracking target 501 shown in FIG. 7). Then, the management unit 351 sets the captured location, the captured date, and the captured time as the detected tracking target information as in the tracking result data 1000 (tracking result information) shown in FIG. , The information associated with the center coordinates (x-coordinate and y-coordinate) of the tracking target is stored in the storage unit 304. For example, in the example of the tracking result data 1000 shown in FIG. 8, the center coordinates of the tracking target corresponding to the captured location “area001”, the date “2016-04-02”, and the time “10:23:45.2”. Of these, the x-coordinate is "0.83" and the y-coordinate is "0.41". The format for expressing the center coordinates may be any format, and may be a normalized value such as the tracking result data 1000 shown in FIG. 8, or an image having a frame of 1920 pixels in width and 1080 pixels in height. If so, the normalized (x, y) = (0.83, 0.41) may be expressed as (x, y) = (1593,442).

The tracking result data 1000 is shown as a CSV (Comma-Separated Values) format file, but the present invention is not limited to this, and the captured location, the captured date, and the captured time are included. , The information may be in any format as long as it is related to the center coordinates (x-coordinate and y-coordinate) of the tracking target.

Next, the locus output unit 310 sets the center coordinates of the tracking target corresponding to the current frame from the tracking result data 1000 stored in the storage unit 304, and the center coordinates of the tracking target corresponding to the frame before the current frame. , Is read. Then, the locus output unit 310 generates a flow line (trajectory line) connecting the center point of the tracking target corresponding to the current frame and the center point of the tracking target corresponding to the frame before the current frame, and individually. It is superimposed on the current frame displayed on the display unit 401 and displayed. In this case, the locus output unit 310 also superimposes and displays the flow lines generated from the frame after the start of the tracking process to the frame before the current frame on the current frame. As a result, as shown in FIG. 7, the individual display unit 401 is superposed on the video data to display the flow line 800 indicating the trajectory of the tracking target after the start of the tracking process. If the purpose is only to store the tracking result information (for example, the tracking result data 1000 shown in FIG. 8), the locus output unit 310 does not necessarily display the flow line 800 on the individual display unit 401 of the tracking screen 400. You don't have to.

By the above operation, the tracking process of the tracking target is performed. The above-mentioned tracking process of the tracking target is supposed to generate tracking result information including the position information of the tracking target by detecting the tracking target from the frame of the video data, but the present invention is not limited to this. .. For example, by giving the tracking target a wireless LAN or a terminal for ultrasonic transmission, or an ID, the date and time when the tracking target is detected, the position information, and the like can be obtained by using a device that receives these position information. May generate tracking result information.

(Determining the standard work area)
FIG. 9 is a flowchart showing an example of a flow of determination processing of a standard work area of the information processing apparatus according to the embodiment. FIG. 10 is a diagram showing an example of a plurality of flow lines of routine work used for obtaining a fixed work area in the information processing apparatus according to the embodiment. FIG. 11 is a diagram showing an example of a state in which a plurality of flow lines of the standard work used for obtaining the standard work area are thickened. 12 and 13 are diagrams showing an example in which the extracted regions are continuous, respectively. FIG. 14 is a diagram illustrating an operation of determining a routine work area in the information processing apparatus according to the embodiment. The process of determining the standard work area by the information processing apparatus 3 according to the present embodiment will be described with reference to FIGS. 9 to 14.

<Step S21>
The first acquisition unit 306 acquires tracking result information corresponding to a plurality of video data received by the video receiving unit 301, received by the data communication unit 321 or stored in the storage unit 304. Then, the process proceeds to step S22.

<Step S22>
The locus output unit 310 connects the points (coordinates of the tracking target) indicated by the tracking result information in chronological order based on the respective tracking result information acquired by the first acquisition unit 306, and displays the flow line on the display unit 353. Display it. Here, it is assumed that the first acquisition unit 306 acquires three tracking result information, and the locus output unit 310 displays three flow lines based on the three tracking result information. Here, FIG. 10 shows three flow lines (flow lines 81 to 813) in which points are connected in time series by the locus output unit 310. The flow lines 81 to 813 indicate the flow lines of routine work, which is a series of work for achieving the purpose of a predetermined unit, respectively. Then, the process proceeds to step S23.

<Step S23>
The thickening unit 322 thickens the flow line displayed by the locus output unit 310. Here, FIG. 11 shows an example in which the flow lines 811 to 813 are thickened by the thickening unit 322, respectively, and the thickening flow lines 811a to 813a are obtained. It should be noted that the width of the thick line section 322 to thicken the flow line may be set by the setting section 303. Then, the process proceeds to step S24.

<Step S24>
The standard extraction unit 323 extracts a region in which the flow lines thickened by the thick line unit 322 overlap by a predetermined number α or more. Here, in the example shown in FIG. 11, it is assumed that the predetermined number α is 2, and the standard extraction unit 323 extracts a region in which two or more of the thick line flow lines 811a to 813a overlap. The predetermined number α may be set by the setting unit 303. Then, the process proceeds to step S25.

<Step S25>
The continuous determination unit 324 determines whether or not the regions extracted by the standard extraction unit 323 are continuous. Here, the fact that the extracted regions are continuous means that, for example, the regions are formed in a ring shape as shown in FIG. 12, or both ends of the region are displayed in the display unit 353 as shown in FIG. It shall indicate the state of being located on each side of the display area where the flow line is displayed. As a result of the determination by the continuous determination unit 324, if the extracted regions are continuous (step S25: Yes), the process proceeds to step S26, and if they are not continuous (step S25: No), the process proceeds to step S27.

<Step S26>
The fixed form determination unit 325 determines a region determined to be continuous by the continuous determination unit 324 as a fixed form work area. Here, in the example of the flow lines 811 to 813, as shown in FIG. 14, the region where two or more of the thick line flow lines 811a to 813a overlap is extracted by the standard extraction unit 323, and the standard determination unit 325 The extracted area is determined as the standard work area 820. Then, the determination process of the standard work area is completed.

<Step S27>
The management unit 351 decrements the predetermined number α by one. Then, the process proceeds to step S28.

<Step S28>
The management unit 351 determines whether or not the predetermined number α is not 1. When α ≠ 1 (step S28: Yes), the process returns to step S24, and the overlapping regions are extracted again with the number of overlapping regions extracted by the standard extraction unit 323 reduced by one. When α = 1 (step S28: No), the process proceeds to step S29.

<Step S29>
When α = 1, the standard work area 325 determines that it is unreliable to set an area that overlaps by 1 as a common area in a plurality of thickened flow lines, and determines the standard work area. It shall not be possible. Then, the determination process of the standard work area is completed.

By the above steps S21 to S29, the routine work area determination process is performed. That is, the standard work area, which is the area where the standard work is performed, is determined by using the model flow line (in the example of FIG. 10, the flow lines 811 to 813).

(Work area determination process)
FIG. 15 is a flowchart showing an example of a flow of determination processing of a work area of the information processing apparatus according to the embodiment. FIG. 16 is a diagram showing an example of a stagnant area and a sudden action area. FIG. 17 is a diagram illustrating an operation of determining a work area in the information processing apparatus according to the embodiment. FIG. 18 is a diagram showing an example of a work area and a moving area determined in the information processing apparatus according to the embodiment. The work area determination process by the information processing apparatus 3 according to the present embodiment will be described with reference to FIGS. 15 to 18.

<Step S41>
The first acquisition unit 306 acquires tracking result information corresponding to a plurality of video data received by the video receiving unit 301, received by the data communication unit 321 or stored in the storage unit 304. Then, the process proceeds to step S42.

<Step S42>
The stagnation extraction unit 326 is a point group indicating each coordinate included in each flow line based on the tracking result information acquired by the first acquisition unit 306, and is a point group in which the tracking target is determined to be stagnation ( The stagnant part) is extracted. Here, as in FIG. 10, the first acquisition unit 306 acquires three tracking result information, and the flow line of the target for which the stagnation extraction unit 326 extracts the stagnation portion is described as a flow line 81 to 813. The stagnation extraction unit 326 calculates, for example, the distance between the nth point indicated by the specific tracking result information and the (n + 1) th point, and if it is less than a preset predetermined value, the (n + 1) th point. The point of is judged to be a stagnant point. Then, the stagnation extraction unit 326 continues to determine whether or not the (n + 2) th point, the (n + 3) th point, and so on are stagnant points, and the (n + m) th point and When the distance to the (n + m + 1) th point is equal to or greater than a predetermined value, it is determined that the (n + 1) th to (n + m) th points group is a stagnant point group (stagnation portion). An example in which the stagnation portion is extracted by the stagnation extraction unit 326 in this way is shown in FIG. Regarding the flow lines 81 to 813, it is assumed that the point group included in the stagnation area 831 to 833 is extracted as the stagnation portion by the stagnation extraction unit 326. Further, in the flow line 813, the sudden action area 835 shows the sudden action of the worker different from the original routine work. Then, the process proceeds to step S43.

<Step S43>
The representative point specifying unit 327 identifies a representative point (representative point) from the points constituting the stagnant portion extracted by the stagnant extraction unit 326 in each of the plurality of flow lines. Specifically, in FIG. 16, the representative point specifying unit 327 is determined by, for example, the average value of the coordinates of the stagnant point groups included in the stagnant areas 831 to 833 among the points constituting the flow line 811. Specify the coordinates as a representative point. That is, in the flow line 811, a representative point is specified for each of the stagnant point groups included in the stagnant areas 831 to 833, and as a result, three representative points are specified. The same applies to the flow lines 812 and 813, and nine representative points are specified for the entire flow lines 81 to 813. Then, the process proceeds to step S44.

<Step S44>
The enlargement unit 328 obtains a distance (moving distance) corresponding to the representative point specified by the representative point identification unit 327. For example, assuming that the distance corresponding to the (n + 1) th point described above is the distance between the nth point and the (n + 1) th point, the (n + 1) th to (n + m) th m points The distance corresponding to the representative point of the point group shall be the average value of the distances corresponding to the points constituting the point group. That is, it is assumed that the average value of the distances corresponding to the (n + 1) th to (n + m) th points is the distance corresponding to the representative point of the (n + 1) th to (n + m) th point group.

Then, the enlargement unit 328 expands the circle around the representative point according to the distance (moving distance) corresponding to the representative point. For example, the enlargement unit 328 greatly enlarges the circle, assuming that the smaller the distance corresponding to the representative point, the greater the degree of stagnation. Here, FIG. 17 shows a state in which the circle is enlarged around the representative points corresponding to the stagnation areas 831 to 833, respectively. Regarding the flow line 811, the expansion region 841a is the representative point of the stagnation portion corresponding to the stagnation region 831 expanded to a circle, and the expansion region 841b is the representative point of the stagnation portion corresponding to the stagnation region 832 expanded to a circle. The expansion region 841c is a circle in which the representative points of the stagnation portion corresponding to the stagnation region 833 are expanded. Regarding the flow line 812, the expansion area 842a is the representative point of the stagnation portion corresponding to the stagnation area 831 expanded to a circle, and the expansion area 842b is the representative point of the stagnation portion corresponding to the stagnation area 832 expanded to a circle. The expansion region 842c is a circle in which the representative points of the stagnation portion corresponding to the stagnation region 833 are expanded. Regarding the flow line 813, the expansion area 843a is the representative point of the stagnation portion corresponding to the stagnation area 833 expanded into a circle, and the expansion area 843b is the representative point of the stagnation portion corresponding to the stagnation area 832 expanded into a circle. The expansion region 843c is a circle in which the representative points of the stagnation portion corresponding to the stagnation region 833 are expanded. The shape of the enlarged portion 328 that expands is not limited to the circle, and may be expanded to other shapes.

Then, the process proceeds to step S45.

<Step S45>
The process extraction unit 329 extracts a region in which representative points expanded by the expansion unit 328 overlap by a predetermined number or more. Here, in the example shown in FIG. 17, it is assumed that the predetermined number is 2, and the process extraction unit 329 extracts a region in which two or more enlarged regions overlap. Specifically, the process extraction unit 329 extracts an overlapping region 850a, which is an overlapping region of two or more of the enlarged regions 841a to 843a corresponding to the stagnant region 831. In addition, the process extraction unit 329 extracts the overlapping region 850b, which is an overlapping region of two or more of the expanded regions 841b to 843b corresponding to the stagnant region 832. Then, the process extraction unit 329 extracts the overlapping region 850c, which is a region in which two or more of the expanded regions 841c to 843c corresponding to the stagnant region 833 are overlapped. The predetermined number may be set by the setting unit 303. Further, the predetermined number may be a different number for each stagnation area. Then, the process proceeds to step S46.

<Step S46>
The process determination unit 330 encloses the area extracted by the process extraction unit 329 (overlapping areas 850a to 850c in the example of FIG. 17) with a rectangle, so that the operator stagnates in the routine work. It is determined as the area (work area) of. It is not essential to enclose the area extracted by the process extraction unit 329 with a rectangle, but by enclosing it with a rectangle and making the work area a polygonal area surrounded by a line segment, as described above. , The area determination unit 342 can easily configure an algorithm for determining whether or not the tracking target exists in the work area. In the example shown in FIG. 17, the process determination unit 330 determines the polygonal work areas 860a to 860c as shown in FIG. 18 by surrounding the overlapping areas 850a to 850c with rectangles, respectively.

Further, the movement determination unit 331 is an area for moving between the work areas, excluding the area overlapping the work area determined by the process determination unit 330 from the standard work area determined by the standard determination unit 325. Determine the area. In the example shown in FIG. 18, the movement determination unit 331 moves the area excluding the area overlapping the work areas 860a to 860c determined by the process determination unit 330 from the standard work area 820 determined by the standard determination unit 325. Determine the area. That is, the process determination unit 330 determines the movement regions 871 to 873 in FIG. 18, respectively.

Further, the movement determination unit 331 is in the movement region on the determined polygon, when the angle of each vertex is less than a predetermined threshold value or greater than or equal to a predetermined threshold value, that is, the bending of the operator in the movement direction indicated by the movement region. When is large, the vertex and the closest vertex among the opposing vertices are connected. Specifically, as shown in FIG. 18, the movement determination unit 331 determines that the angle of the apex 881 is less than a predetermined threshold value in the movement region 871, and sets it as the closest apex among the vertices facing the apex 881. To form a dividing line 871a. Similarly, the movement determination unit 331 determines that the angle of the apex 883 is less than a predetermined threshold value in the movement region 873, and connects the nearest apex among the vertices facing the apex 883 to form a dividing line 873a. .. By forming a dividing line in the moving area in this way, it is possible to indicate a portion of the moving area where the worker has a large degree of bending in the moving direction. A large degree of bending means that some anomalous behavior may have occurred during the movement between normal processes, and by forming a dividing line, the anomalous behavior may have occurred. Can contribute to the improvement of routine work. Then, the work area determination process is completed.

By the above steps S41 to S46, the work area determination process is performed. That is, the work area, which is the area of the process in which the worker is stagnant in the routine work, is determined by using the flow line as a model (in the example of FIG. 10, the flow lines 811 to 813).

As described above, using the acquired flow line information (tracking result information), the stagnant part is extracted from the points constituting the flow line, the specific points constituting the stagnant part are expanded, and the points are expanded. The work area is determined based on the area where the enlarged areas overlap by a predetermined number or more. As a result, it is not necessary to draw a flow line once and manually set the work area while looking at the flow line, and the work area can be determined directly from the acquired flow line. Further, as a result, the work area can be automatically determined, which can contribute to speeding up the improvement of routine work.

In the flowchart shown in FIG. 15, the representative point identification unit 327 identifies the representative point from the points constituting the stagnation portion extracted by the stagnation extraction unit 326, and the expansion unit 328 expands the representative point to expand the area. However, it is not limited to this. That is, all the points constituting the stagnation portion extracted by the stagnation extraction unit 326 may be expanded, regions overlapping by a predetermined number or more may be extracted, and the work area may be determined based on the regions.

Further, the tracking process of the tracking target (generation of tracking result information) shown in FIGS. 7 and 8 described above, the routine work area and the determination process of the work area shown in FIGS. 9 and 15, and the second acquisition unit. The information processing device 3 is responsible for all the abnormality determination processing of the worker's work based on the functions of the area determination unit 342, the measurement unit 343, the time determination unit 344, and the notification unit 345 for the real-time video data acquired by 341. However, it is not limited to this. That is, these processes may be distributed and processed by separate devices. For example, the tracking process of the tracking target may be performed by a device different from the information processing device 3, and the information processing device 3 may perform the routine work area and the work area determination process and the worker's work abnormality determination process. Good.

It should be noted that the setting unit 303, the reproduction unit 305, the first acquisition unit 306, the designation unit 307, the registration unit 308, the detection unit 309, the locus output unit 310, the data communication unit 321 and the thickening unit 322 of the above-described embodiment are standard. Extraction unit 323, continuous determination unit 324, standard determination unit 325, stagnation extraction unit 326, representative point identification unit 327, enlargement unit 328, process extraction unit 329, process determination unit 330, movement determination unit 331, second acquisition unit 341, When at least one of the function units of the area determination unit 342, the measurement unit 343, the time determination unit 344, the management unit 351 and the display control unit 352 is realized by executing the program, the program is preliminarily incorporated in the ROM or the like. Will be provided. Further, the program executed by the information processing apparatus 3 of the above-described embodiment is a file in an installable format or an executable format, and is a CD-ROM (Computer Disk Read Only Memory), a flexible disk (FD), or a CD-. It may be configured to be recorded and provided on a computer-readable recording medium such as an R (Compact Disk Recordable) or a DVD. Further, the program executed by the information processing apparatus 3 of the above-described embodiment may be stored on a computer connected to a network such as the Internet and provided by downloading via the network. Further, the program executed by the information processing apparatus 3 of the above-described embodiment may be configured to be provided or distributed via a network such as the Internet. Further, the program executed by the information processing apparatus 3 of the above-described embodiment has a module configuration including at least one of the above-mentioned functional units, and as the actual hardware, the CPU programs from the above-mentioned ROM. By reading and executing the above-mentioned parts, the above-mentioned parts are loaded on the main storage device and generated.

1 Image processing system 2, 2a to 2d Imaging device 3 Information processing device 4 Hub 10 External device 101 CPU
102 ROM
103 RAM
104 External storage 105 Display 106 Network I / F
107 Keyboard 108 Mouse 109 DVD Drive 110 DVD
111 External device I / F
112 Speaker 113 Bus 301 Video receiving unit 302 Input unit 303 Setting unit 304 Storage unit 305 Playback unit 306 First acquisition unit 307 Designation unit 308 Registration unit 309 Detection unit 310 Trajectory output unit 321 Data communication unit 322 Thickening unit 323 Standard extraction unit 324 Continuous judgment unit 325 Standard determination unit 326 Stagnation extraction unit 327 Representative point identification unit 328 Expansion unit 329 Process extraction unit 330 Process determination unit 331 Movement determination unit 341 Second acquisition unit 342 Area determination unit 343 Measurement unit 344 Time determination unit 345 Notification Part 351 Management part 352 Display control part 353 Display part 400 Tracking screen 401 Individual display part 401a Camera selection tab 403 Video operation button part 404 Seek bar 405 Slider 406 Enlargement / reduction button part 407 Span change button part 408 Time display part 409 Registration button 410 Tracking Start button 411 Tracking stop button 500 Worker 501 Tracking target 550 Designated area 800 Flow line 811-813 Flow line 811a to 813a Thickened flow line 820 Routine work area 831 to 833 Stagnation area 835 Sudden action area 841a to 843a, 841b 843b, 841c to 843c Enlarged area 850a to 850c Overlapping area 860a to 860c Working area 871 to 873 Moving area 871a, 873a Separator 881, 883 Pole 1000 Tracking result data

JP-A-2016-042306

Claims (11)

A first acquisition unit that acquires a plurality of tracking result information including position information of a tracking target in a routine work which is a series of operations of a predetermined unit, and
With the first extraction unit that extracts the stagnant portion, which is a group of stagnant points, based on the moving distance of the points indicated by the position information of the tracking target of the plurality of tracking result information acquired by the first acquisition unit. ,
An expansion unit for obtaining an expansion area obtained by expanding a predetermined point based on the stagnation portion corresponding to each tracking result information, and an expansion unit.
A first determination unit that determines a work area based on an area in which at least one of the plurality of expansion areas obtained by the expansion unit overlaps.
Information processing device equipped with. A specific unit that identifies a representative point that is a point at a specific position based on the position of the point included in the stagnant portion corresponding to each tracking result information extracted by the first extraction unit is further provided.
The information processing device according to claim 1, wherein the enlarged portion obtains the enlarged region in which the representative point corresponding to each stagnant portion is enlarged. The information processing device according to claim 1, wherein the expansion unit expands a point included in the stagnation portion corresponding to each tracking result information to obtain the expansion area. In each of the plurality of tracking result information acquired by the first acquisition unit, a flow line generation unit that combines points indicating the position information of the tracking target to form a flow line.
A thickened portion that thickens the flow line to a predetermined width,
A second extraction unit that extracts a region in which the flow lines thickened by the thick line portion overlap by a predetermined number or more,
The first determination unit for determining whether or not the regions extracted by the second extraction unit are continuous, and the region determined to be continuous by the first determination unit are the regions where the routine work is performed. The second decision unit that determines the standard work area, which is
The information processing apparatus according to any one of claims 1 to 3, further comprising. When the first determination unit determines that the regions extracted by the second extraction unit are not continuous, a change unit for reducing the predetermined number is further provided.
The information processing apparatus according to claim 4, wherein the second extraction unit extracts a region in which the flow lines thickened by the thickening unit overlap with the predetermined number reduced by the changing unit. The area excluding the area overlapping the work area determined by the first determination unit from the standard work area determined by the second determination unit is determined as a movement area which is an area for moving between the work areas. The information processing apparatus according to claim 4 or 5, further comprising a third determination unit. A second determination unit that determines whether or not the tracking target exists in the work area,
When the second determination unit determines that the tracking target exists in the work area, the work time as the time existing in the work area is measured, and the maximum value, the minimum value, and the average value of the work time are measured. A measuring unit that calculates at least one of them,
The information processing apparatus according to any one of claims 1 to 6. A third determination unit that determines whether or not the work time measured by the measurement unit is within a predetermined range,
When the third determination unit determines that the work time is not within the predetermined range, the notification unit for notifying the determination result and the notification unit.
The information processing apparatus according to claim 7, further comprising. With one or more imaging devices
The information processing device according to any one of claims 1 to 8.
With
The information processing device
A second acquisition unit that acquires a frame from each video data captured by one or more imaging devices, and
A detection unit that detects a tracking target from the frame acquired by the second acquisition unit, and
A generation unit that generates the tracking result information including the position information of the tracking target from the detection result of the tracking target by the detection unit.
Information processing system equipped with. The first acquisition unit, the first extraction unit, the enlargement unit, and the first determination unit are mounted on the information processing device.
The acquisition step in which the first acquisition unit acquires a plurality of tracking result information including the position information of the tracking target in the routine work which is a series of operations of a predetermined unit, and
An extraction step of the first extraction unit, based on the movement distance of the point indicated by the position information of the tracking object of the acquired plurality of tracking result information, it extracts a stagnant portion is the point group has stagnated,
An expansion step in which the expansion unit obtains an expansion area in which a predetermined point based on the stagnation portion corresponding to each tracking result information is expanded.
A determination step in which the first determination unit determines a work area based on an area where at least one of the plurality of enlarged areas overlaps.
Information processing method having. For information processing equipment
Acquisition processing to acquire a plurality of tracking result information including the position information of the tracking target in the routine work which is a series of operations of a predetermined unit, and
An extraction process for extracting a stagnant portion, which is a group of stagnant points, based on the moving distance of points indicated by the position information of the tracked target in the acquired plurality of tracking result information.
An expansion process for obtaining an expansion area obtained by expanding a predetermined point based on the stagnant portion corresponding to each tracking result information, and
A determination process for determining a work area based on an area where at least one of the plurality of enlarged areas overlaps.
A program to execute.

Images