JP2020136954A - Processing device, processing program, and processing method - Google Patents

Abstract

To improve a video image confirmation operability of a photographing object by automatically reproducing each reproduction image by setting an optimal synchronous difference when a plural kind of video images photographed at a plurality of locations in accordance with the movement of the photographing object is simultaneously reproduced on a view of a different direction.SOLUTION: A monitoring processing part 32 performs a monitoring while reproducing a moving image on the basis of a pre-stored individual reproduction time, and therefore, a moving image reproduction time can be set as a sufficient requirement time, and can improve operability. As a comparison example, when performing the monitoring for a fixed time (e.g., 10 minutes of a default value), a stand-by time becomes long when a monitoring person monitors an article 14 with a plurality of moving images, and the operability is reduced. On the other hand, since an appropriate reproduction time tk is determined each monitoring person, the stand-by time can be reduced, and the monitoring of the article 14 can be performed with the sufficient requirement time.SELECTED DRAWING: Figure 5

Description

The present invention relates to a processing apparatus, a processing program, and a processing method.

Conventionally, when video surveillance is applied to a specific area such as a production line in a factory or a delivery sorting system in a distribution center, Patent Document 1 describes a flow in a plurality of monitoring locations as a monitoring means for simultaneously displaying a plurality of videos. It is described that the state of one object to be moved at the time of passing through each camera is reproduced and visually confirmed. According to Patent Document 1, it is possible to detect a plurality of time points to be reproduced.

Japanese Unexamined Patent Publication No. 2018-19373

However, in the technique described in Patent Document 1, since the selection of a plurality of types of images to be simultaneously reproduced and the designation of the reproduction time in the image are left to the manual operation of the operator, the image confirmation work is complicated. It has become.

According to the present invention, when a plurality of types of images taken at a plurality of locations are simultaneously reproduced on a line of sight in different directions due to the movement of an object to be photographed, each reproduced image is automatically reproduced by setting an optimum synchronization difference. It is an object of the present invention to obtain a processing device capable of improving the image confirmation workability of an object to be photographed.

The processing device of the present invention is a process in which a viewer visually monitors an image simultaneously reproduced and displayed on a display device based on a plurality of types of image information captured by a plurality of photographing devices in a predetermined photographing area while moving the line of sight. It is a device, and based on the timing information associated with each of the plurality of types of shooting information, the playback display start timing on the display device is shifted, and each image based on the plurality of types of image information is displayed and controlled. It has a display control unit.

The processing program of the present invention is a computer that visually monitors an image simultaneously reproduced and displayed on a display device by a viewer while moving the line of sight, based on a plurality of types of image information obtained by photographing a predetermined imaging area with a plurality of imaging devices. , A display control for displaying and controlling each image based on the plurality of types of image information by shifting the playback display start timing on the display device based on the timing information associated with each of the plurality of types of shooting information. Make it function as a department.

The processing method of the present invention is a process in which a viewer visually monitors an image simultaneously reproduced and displayed on a display device based on a plurality of types of image information captured by a plurality of imaging devices in a predetermined imaging area while moving the line of sight. It is a method, and based on the timing information associated with each of the plurality of types of shooting information, the playback display start timing on the display device is shifted, and each image based on the plurality of types of image information is displayed and controlled. To do.

According to the present invention, when a plurality of types of images shot at a plurality of locations are simultaneously reproduced on a line of sight in different directions due to the movement of an object to be photographed, each reproduced image is automatically reproduced by setting an optimum synchronization difference. As a result, the effect of improving the workability of checking the image of the object to be photographed can be obtained.

It is a perspective view which shows the article transport line which concerns on this embodiment. (A) is a schematic diagram of a surveillance system using a surveillance camera according to the present embodiment, and (B) is a conceptual diagram of an individual table stored in a large-scale storage device. It is a functional block diagram of a monitoring system mainly composed of a reproduction time determination processing unit. It is a moving image reproduction transition diagram of the monitor which illustrated the case where four surveillance cameras are divided into four and reproduced at the same time in the reproduction time determination processing unit. It is a functional block diagram of a monitoring system mainly composed of a monitoring processing unit. It is a moving image reproduction transition diagram of the monitor which illustrated the case where four surveillance cameras are divided into four and reproduced at the same time in the monitoring processing unit. It is a control flowchart which shows the monitoring system main routine executed in the main control part which concerns on this embodiment. It is a control flowchart which shows the reproduction time determination processing routine in the reproduction time determination processing unit which is executed by the instruction from the main control unit. It is a control flowchart which shows the monitoring processing routine in the monitoring processing part which is executed by the instruction from the main control part.

FIG. 1 is a perspective view showing an article transport line 10 according to the present embodiment.

The goods transport line 10 can be applied to a production line in a factory, a delivery sorting system in a distribution center, and the like.

The article transport line 10 according to the present embodiment includes the main trunk line 12 in a straight line. A belt conveyor 12A is laid on the main trunk road 12, and a drive source (not shown) is used to move the main trunk road 12 while looping in an endless manner.

Therefore, when the article 14 is conveyed from, for example, an article stocker (not shown) provided on the upstream side of the main trunk road 12, as shown by an arrow A in FIG. 1, the article 14 is transferred to the belt conveyor 12A. As it moves, it is sent downstream of the main trunk road 12 as shown by the arrow B in FIG.

In the present embodiment, the belt conveyor 12A of the main trunk road 12 moves at a constant speed, and the plurality of articles 14 to be transported move at a constant interval and at a constant speed. There is.

A tolerance ± α is set for the transport interval of the article 14, and as a result, the transport interval of the article 14 includes a case where the tolerance varies within the tolerance ± α.

Further, the transport speed may also have a predetermined tolerance ± β.

As shown in FIG. 1, a plurality of (two in FIG. 1) branch roads 16A and 16B are provided from the main trunk road 12.

A switching guide unit (not shown) is attached to the boundary between the main trunk road 12 and the branch roads 16A and 16B, and the direction can be changed from the main trunk road 12 to the branch roads 16A and 16B according to the type of the article 14. It has become like. Such branch roads 16A and 16B can be applied to the delivery sorting of the distribution center, and as an example, the terminal of the main trunk road 12 and the transport truck waiting at the end of the branch roads 16A and 16B for each delivery area ( It is possible to sort into (not shown).

Here, in the article transport line 10 of the present embodiment, the surveillance camera 18 is attached above the main trunk road 12 and the branch roads 16A and 16B. The number L of the surveillance cameras 18 is not particularly limited, but in FIG. 1, five surveillance cameras 18 are arranged on the main road 12 and two surveillance cameras 18 are arranged on the branch road 16A. , One surveillance camera 18 is arranged on the branch road 16B.

Here, the arrangement state will be described by paying attention to four surveillance cameras 18 (shown by solid lines in FIG. 1) arranged on the main trunk road 12.

The four surveillance cameras 18 are arranged at regular intervals. The arrangement interval of the surveillance cameras 18 does not depend on the transportation interval of the articles 14, and may be set independently. Further, it does not have to be arranged at regular intervals.

Each surveillance camera 18 will take a picture of the moving article 14 on the main road 12 from above, and the image will be a flat image (see FIG. 2B). A part or all of the surveillance camera 18 may be tilted to take a deep image (so-called bird's-eye view image) instead of a flat image.

FIG. 2A shows a surveillance system 20 using the surveillance camera 18 according to the present embodiment.

The monitoring system 20 includes a main control unit 22. The main control unit 22 includes a CPU 22A, a RAM 22B, a ROM 22C, an input / output unit (I / O) 22D, and a data bus connecting these includes a bus 22E such as a control bus. The monitoring system 20 collectively manages the video processing unit 28, the playback time determination processing unit 30, and the monitoring processing unit 32, which will be described later, by a processing program by the CPU 22A.

A user interface (keyboard, mouse, display, etc.) 24 and a video processing unit 28 are connected to the I / O 22D.

Further, the reproduction time determination processing unit 30 and the monitoring processing unit 32 are connected to the I / O 22D. The details of the reproduction time determination processing unit 30 and the monitoring processing unit 32 will be described later, but each processing is executed by receiving a processing execution instruction via the user interface 24. In addition, the user interface 24 manages the progress of processing of each part.

Further, a large-scale storage device 26 is connected to the I / O 22D, and it is possible to store information necessary for processing executed by the monitoring system 20. As the main information thereof, the large-scale storage device 26 stores a reproduction time table (individual table 26T) set for each observer by the reproduction time determination processing unit 30 (FIG. 2B). (See). The individual table 26T will be described later together with a detailed description of the reproduction time determination processing unit 30.

(Video processing unit 28)

As shown in FIGS. 3 and 5, the video processing unit 28 includes a plurality of surveillance cameras 18 (surveillance camera group 18G), and the video (mainly moving image) captured by each surveillance camera 18 is stored in the video information storage unit 34. It is supposed to be accumulated.

Further, the video processing unit 28 includes a monitor 36. A playback processing unit 38 is connected to the monitor 36, and the playback processing unit 38 takes in the video information stored in the video information storage unit 34 based on, for example, the identification information that identifies the video information, and the monitor 36 To play and display.

Although details will be described later, the monitor 36 of the video processing unit 28 is equipped with an eye tracking camera 40 applied by the reproduction time determination processing unit 30.

Here, the screen of the monitor 36 is divided so that the images of the plurality of surveillance cameras 18 are simultaneously displayed (reproduced) (see FIGS. 4 and 6).

The number of divided screens may be the number of surveillance cameras 18 installed, but depending on the size of the monitor 36, all the surveillance cameras 18 may be divided and displayed.

In the present embodiment, as shown in FIGS. 4 and 6, the images of the four surveillance cameras 18 are displayed at the same time. That is, the images taken by the four surveillance cameras 18 on the main trunk road 12 shown by the solid line in FIG. 1 are divided into four screens of the monitor 36 and displayed at the same time.

For example, the images of the other surveillance cameras 18 (shown by the dotted line in FIG. 1) are displayed on the monitor 36 at different times, and by repeating the time-division display, the images of all the surveillance cameras 18 are constant. It can be monitored in time.

In the present embodiment, when confirming the past transportation state of the article 14, the monitoring processing unit 32 for reproducing the image captured (recorded) at the past time point is provided.

The monitoring processing unit 32 reproduces images of the status of the article 14 (for example, the moving state of the article 14 (single) subject to failure) taken by a plurality of surveillance cameras 18 along the transport path.

For example, the four surveillance cameras 18 on the main road 12 shown by the solid line in FIG. 1 may display an image of the same moving article 14 on the belt conveyor 12A.

In this case, the interval (time of the observer's line of sight or face movement) from confirming one image to seeing the next image varies depending on the observer, and four images are displayed at what timing. It is necessary to specify whether (whether to determine the playback time).

Therefore, as the monitoring system 20 of the present embodiment, the playback time determination processing unit 30 that determines the playback time by determining the movement of the observer's line of sight or face by the eye tracking camera 40 (see FIG. 3), and the playback Time determination processing unit 30 A monitoring processing unit 32 for monitoring the state of the specific article 14 is provided based on the determined reproduction time.

(Reproduction time determination processing unit 30)

FIG. 3 shows a functional block diagram of the monitoring system 20 mainly composed of the reproduction time determination processing unit 30. The hardware configuration of each block shown in FIG. 3 is not limited, and a part or all of the blocks may be operated by a processing program.

When the main control unit 22 gives a process execution instruction to the execution management unit 42 of the reproduction time determination processing unit 30, the execution management unit 42 issues a monitoring instruction to the synchronization difference default value reading unit 44 and the line-of-sight extraction unit 46. Notice.

The synchronous difference default value reading unit 44 is connected to the synchronous difference default value storage unit 48, and receives a start instruction from the execution management unit 42 to store the synchronous difference default value stored in the synchronous difference default value storage unit 48. It is read out and sent to the reproduction processing unit 38 of the video processing unit 28.

The reproduction processing unit 38 reads out the image of the specific article 14 (see FIG. 1) from the image information accumulated by the image information taken by the plurality of surveillance cameras 18 (surveillance camera group 18G). In the present embodiment, the video information of the specific article 14 taken by the four surveillance cameras 18 arranged on the main trunk road 12 is read out, and the monitor 36 is based on the synchronization difference default value (here, 10 seconds). Display on.

At this time, the image of each divided display area on the monitor 36 is put into a paused state with the cueing process applied, and as shown in FIG. 4, the divided display area 1 (upper left) is used as a reference, and the divided display area is used as a reference. The video in the split display area 2 (upper right) is played 10 seconds after the start of playback of the video 1 and the video in the split display area 3 (lower left) is played 10 seconds after the start of playback in the split display area 2. The moving image in the divided display area 4 (lower right) is played 10 seconds after the start of playback of 3.

On the other hand, when the line-of-sight extraction unit 46 receives a start instruction from the execution management unit 42, it acquires information on the movement of the observer's line of sight (or face) from the eye tracking camera 40 and recognizes the line-of-sight information. In the present embodiment, the eye tracking camera 40 is arranged at a position where the optical axis faces the monitor (for example, in the vicinity of the monitor 36), but when executing the playback time determination process, the eye tracking glasses or the like are worn on the monitor. It may be attached.

The line-of-sight information extracted by the line-of-sight extraction unit 46 is sent to the reproduction time determination unit 50. The reproduction time determination unit 50 specifies the movement time of the line of sight between the divided display areas based on the line-of-sight information, and the time tk for gazing at one divided display area 1 to 4 (here, four surveillance cameras 18). T1 to t4) are determined, and the first counter 52 and the moving image reproduction time table generation unit 54 are notified.

The first counter 52 is connected to the reproduction processing unit 38, and instructs to repeat the acquisition of the line-of-sight information N1 times (for example, about 10 to 50 times). By repeating this, the accuracy of the line-of-sight information can be improved.

On the other hand, the moving image reproduction time table generation unit 54 temporarily cumulatively stores the reproduction time for N1 times received from the reproduction time determination unit 50, and based on the end notification from the first counter 52, the moving image reproduction time table is stored. Generate. For example, as the moving image playback time table, the average value, the median value, or the like of the cumulatively stored N1 playback time can be applied.

The video playback time table generated by the video playback time table generation unit 54 is sent to the large-scale storage device 26 of the main control unit 22, and the video playback time table (individual table 26T) for each target person (monitorer) of the line-of-sight information. ) Is memorized.

The individual table 26T stored in the large-scale storage device 26 is transferred to the monitoring processing unit 32 and used during the monitoring processing in the monitoring processing unit 32.

(Reproduction processing unit 38)

FIG. 5A shows a functional block diagram of the monitoring system 20 mainly composed of the monitoring processing unit 32. The hardware configuration of each block shown in FIG. 5 is not limited, and a part or all of the blocks may be operated by a processing program.

When the monitoring mode execution instruction receiving unit 56 of the monitoring processing unit 32 receives the execution of the process from the main control unit 22, the monitoring mode execution instruction receiving unit 56 sets the personal identification information acquisition unit 58 and the moving image playback time table reading unit 60. Start it.

The personal identification information acquisition unit 58 acquires the personal identification information of the observer to be monitored and notifies the video playback time table acquisition unit 62.

The moving image playback time table acquisition unit 62 notifies the main control unit 22 of the personal identification information, and the main control unit 22 is based on the personal identification information from the individual table 26T of the large-scale storage device 26. , The moving image reproduction time table is extracted and transferred to the moving image reproduction time table acquisition unit 62.

The moving image reproduction time table acquisition unit 62 temporarily stores the acquired moving image reproduction time table (see FIG. 5B) in the table temporary storage unit 64.

On the other hand, when the moving image reproduction time table reading unit 60 receives an activation instruction from the monitoring mode execution instruction receiving unit 56, the moving image reproduction time table reading unit 60 acquires the moving image reproduction time table temporarily stored in the table temporary storage unit 64 and obtains the second counter. It is sent to the reproduction processing unit 38 via 66. The transmission of this moving image reproduction time table serves as one reproduction instruction.

At this time, the image of each divided display area on the monitor 36 is put into a paused state with the cueing process applied, and as shown in FIG. 6, the divided display area 1 (upper left) is used as a reference, and the divided display area is used as a reference. The video in the split display area 1 (upper left) ends and the video in the split display area 2 (upper right) starts playing t1 seconds after the start of playback of the moving image 1 and the split display area 2 starts playing t2 seconds later. The video in the area 2 (upper left) ends playback, the video in the split display area 3 (lower left) starts playing, and the video in the split display area 3 (lower left) ends after t3 after the start of playback in the split display area 3. The moving image in the split display area 4 (lower right) starts playing, and the moving image in the split display area 4 (lower right) ends after t4 seconds.

The second counter 66 acquires the number of reproductions from the reproduction processing unit 38, and repeats the transmission of the moving image reproduction time table as many times as a predetermined number of reproductions.

The operation / action of the present embodiment will be described below with reference to the flowcharts of FIGS. 7 to 9.

FIG. 7 is a control flowchart showing a monitoring system main routine executed by the main control unit 22 according to the present embodiment.

In step 100, it is determined from the user interface 24 whether or not there is an instruction to execute the process. If a negative determination is made in step 100, the routine ends.

If an affirmative decision is made in step 100, the process proceeds to step 102 to determine the processing mode.

If the processing mode is the reproduction time determination process in step 102, the process proceeds to step 104, and the reproduction time determination processing unit 30 is controlled to execute the reproduction time determination process.

If the processing mode is the monitoring process in step 102, the process proceeds to step 106 to control the monitoring processing unit 32 and execute the monitoring process.

(Playback time determination process)

FIG. 8 is a control flowchart showing a reproduction time determination processing routine in the reproduction time determination processing unit 30 executed by an instruction from the main control unit 22.

In step 150, the set value of the first counter 52 is N1, and the number of surveillance cameras is L. In the present embodiment, the set value N1 of the first counter 52 is preferably 10 to 50. There is no particular limitation on the count value, but for example, when the line-of-sight repetition average value is used to determine the playback time, an appropriate average value cannot be obtained without data for at least 10 times, and even if it is performed 50 times or more. From the viewpoint that the accuracy of the average value does not increase, N1 = 10 to 50 was set as an example.

On the other hand, the number of surveillance cameras L simultaneously reproduces (displays) four surveillance cameras 18 (shown by solid lines in FIG. 1) arranged linearly along the main trunk road 12 shown in FIG. 1 on the monitor 36. Set as a thing. The number of surveillance cameras is not particularly limited, but it is necessary and sufficient as a distance for monitoring the article 14 on the belt conveyor 12A, and when the screen of the monitor 36 is divided, the number of surveillances is insufficient in the 2 and 3 divisions, and 5 In the case of division or more, L = 4 is set as an example from the viewpoint that the display unit size becomes smaller when a monitor of a general size (about 30 inches) is assumed (see FIG. 4).

In the next step 152, 0 is set as the initial value of the variable i, then the process proceeds to step 154, 0 is set as the initial value of the variable k, and the process proceeds to step 156.

In step 156, the variable i is incremented (i = i + 1), and then in step 158, the playback video is cueed and paused in all the divided display areas (4 areas in FIG. 4). As a state, the process proceeds to step 160.

In step 160, the variable k is incremented (k = k + 1), and the process proceeds to step 162.

In the next step 162, the moving image in the divided display area k is started to be played back, and the process proceeds to step 164. The maximum playback time is td (default 10 seconds).

In step 164, it is determined whether or not the observer's line of sight or face has moved from the divided display area k to k + 1 based on the line of sight information from the eye tracking camera 40.

If a negative determination is made in step 164, the current display state is maintained, and if an affirmative determination is made, the process proceeds to step 166, and the playback time of the moving image in the i-th divided display area k (the observer plays the moving image). (Time spent watching) tk is acquired, and the process proceeds to step 168.

In step 168, the operation of the divided display area k ends reproduction, and the process proceeds to step 170. In step 168, the default value of td time reproduction may be continued. In this case, even if the observer moves his / her line of sight, he / she waits for the remaining time of td until the reproduction starts in the next divided display area.

In step 170, it is determined whether or not the variable k has reached L (k = L), and if a negative determination is made, the process returns to step 160 and the above steps are repeated.

If an affirmative judgment is made in step 170, the process proceeds to step 172 to determine whether the variable i has reached N1 (i = N1), and if a negative judgment is made, the process returns to step 156. The above process is repeated.

If an affirmative decision is made in step 172, the process proceeds to step 174 to determine the moving image reproduction time tk in the divided display areas 1 to L. In this case, it can be the average value or the median value of the information obtained by repeating i times.

In step 174, all the past information is stored, and when the information becomes enormous, the change in the line-of-sight information of each individual observer is learned and monitored instead of the simple average value and median value. The appropriate reproduction time tk may be predicted by the observer in consideration of the time zone, the degree of fatigue, and the like.

In the next step 176, the determined reproduction time tk is stored in the large-scale storage device 26 of the main control unit 22, and this routine ends.

When the reproduction time determination processing unit 30 is an independent reproduction time determination processing device, the independent reproduction time determination processing device may be provided with a storage unit for storing the reproduction time tk.

(Transition of video playback on the monitor to determine the playback time)

FIG. 4 is a moving image reproduction transition diagram of the monitor 36, exemplifying a case where the four surveillance cameras 18 are divided into four and reproduced at the same time.

FIG. 4A shows a moving image playback state on the monitor 36 when all the divided display areas are cueed and paused, and k = 1. In this case, as shown in FIG. 4A, the upper left split display area is played back for td seconds (10 seconds by default), and the other split display areas (upper right, lower left, and lower right) are in the paused state. (Still image).

Here, based on the line-of-sight information from the eye tracking camera 40, the playback time determination unit 50 measures the time t1 from the start of moving image playback in the upper left split display area to the movement of the observer's line of sight to the upper right split display area. To do. It should be noted that not all observers move directly from the upper left to the upper right divided display area, and the line of sight may be temporarily turned away from the monitor 36. Therefore, the time t1 until the monitor leaves the upper left divided display area may be set. ..

When the default 10 seconds have elapsed, as shown in FIG. 4B, the split display area on the upper right is played back for td seconds (10 seconds as the default), and the split display area on the upper left is in the display end state (so-called sandstorm, Blackout, whiteout, still image at the end of playback, or the next cueing state, etc.) and other divided display areas (lower left and lower right) are in the paused state (still image).

Here, based on the line-of-sight information from the eye tracking camera 40, the playback time determination unit 50 measures the time t2 from the start of moving image playback in the upper right split display area to the movement of the observer's line of sight to the lower left split display area. To do. It should be noted that not all observers move directly to the split display area from the upper right to the lower left, and the line of sight may be temporarily turned away from the monitor 36. Therefore, the time t2 until the monitor leaves the split display area on the upper right may be set. ..

When the default 10 seconds have elapsed, as shown in FIG. 4C, the lower left split display area is played back for td seconds (10 seconds as the default), and the upper left and upper right split display areas are in the display end state (so-called). The sandstorm, blackout, whiteout, still image at the end of playback, or the next cueing state, etc.) and other divided display areas (lower right) are in the paused state (still image).

Here, based on the line-of-sight information from the eye tracking camera 40, the playback time determination unit 50 sets the time t3 from the start of moving image playback in the lower left split display area to the movement of the observer's line of sight to the lower right split display area. measure. It should be noted that not all observers move directly from the lower left to the lower right split display area, and the line of sight may be once distracted from the monitor 36. Therefore, the time t3 until the monitor leaves the lower left split display area may be set. Good.

When the default 10 seconds have elapsed, as shown in FIG. 4D, the lower right split display area is played back for td seconds (10 seconds as the default), and the upper left, upper right, and lower left split display areas are in the display end state. (So-called sandstorm, blackout, whiteout, still image at the end of playback, or the next cueing state, etc.).

Here, based on the line-of-sight information from the eye tracking camera 40, in the playback time determination unit 50, the time t4 from the start of moving image playback in the lower right split display area to the movement of the observer's line of sight to the upper left split display area. To measure. It should be noted that not all observers move directly from the lower right to the upper left split display area, and the line of sight may be once distracted from the monitor 36. Therefore, the time t4 until the viewer leaves the lower right split display area is set. May be good.

The variables i are counted from FIGS. 4 (A) to 4 (D), and the reproduction determination time tk of each divided display area is determined (tk = t1, t2, t3, t4).

(Monitoring process)

FIG. 9 is a control flowchart showing a monitoring processing routine in the monitoring processing unit 32 executed by an instruction from the main control unit 22.

In step 200, as an initial setting, the set value of the second counter 66 is set to N2, the number L of the surveillance cameras 18 is set to 4, and the process proceeds to step 202.

Since the set value of the second counter 66 depends on the number of articles to be inspected, it is not numerically referred to and is at least 1 or more (a positive integer). Further, the surveillance camera 18 sets L = 4 corresponding to the reproduction time determination processing routine shown in FIG.

In step 202, 0 is set as the initial value of the variable j, then the process proceeds to step 204, 0 is set as the initial value of the variable k, and the process proceeds to step 206.

In step 206, the reproduction time table of the observer is read from the large-scale storage device 26 of the main control unit 22 based on the personal identification information of the observer, and the process proceeds to step 208. When the monitoring processing unit 32 is an independent reproduction time determination processing device, a storage unit for storing the reproduction time tk may be provided in the independent monitoring processing.

In step 208, the variable j is incremented (j = j + 1), and then in step 210, the playback moving image is cueed and paused in all the divided display areas (four areas in FIG. 6). As a state, the process proceeds to step 212.

In step 212, the variable k is incremented (k = k + 1), and the process proceeds to step 214.

In step 214, the jth moving image of the divided display area k is started to be played back, and the process proceeds to step 216. In step 216, it is determined whether or not tk seconds have elapsed since the start of reproduction.

If a negative determination is made in step 216, the current display state is maintained, and if an affirmative determination is made, the process proceeds to step 218, the jth video of the divided display area k is played back, and the process proceeds to step 220. To do.

In step 220, it is determined whether or not the variable k has reached L (k = L), and if a negative determination is made, the process returns to step 212 and the above steps are repeated.

If an affirmative judgment is made in step 220, the process proceeds to step 222 to determine whether the variable j has reached N2 (j = N2), and if a negative judgment is made, the process returns to step 208. The above process is repeated.

If an affirmative decision is made in step 222, this routine ends.

(Transition of monitor video playback for monitoring processing)

FIG. 6 is a moving image reproduction transition diagram of the monitor 36, exemplifying a case where the four surveillance cameras 18 are divided into four and simultaneously reproduced.

FIG. 6A shows a moving image playback state on the monitor 36 when all the divided display areas are cueed and paused, and k = 1. In this case, as shown in FIG. 6A, the upper left split display area is played back for t1 seconds, and the other split display areas (upper right, lower left, and lower right) are in the paused state (still image). ..

When t1 second elapses, as shown in FIG. 6B, the split display area on the upper right is played back for t2 seconds, and the split display area on the upper left is in the display end state (so-called sandstorm, blackout, whiteout, playback end). The still image at the time, the next cueing state, etc.) and other divided display areas (lower left and lower right) are in the paused state (still image).

When t2 seconds have elapsed, as shown in FIG. 6C, the lower left split display area is played back for t3 seconds, and the upper left and upper right split display areas are in the display end state (so-called sandstorm, blackout, whiteout, etc.). The still image at the end of playback, the next cueing state, etc.) and other divided display areas (lower right) are in the paused state (still image).

When t3 seconds have elapsed, as shown in FIG. 6D, the lower right split display area is played back for t4 seconds, and the upper left, upper right, and lower left split display areas are in the display end state (so-called sandstorm, blackout). , Whiteout, still image at the end of playback, or the next cueing state, etc.).

The above FIGS. 6A to 6D are regarded as one count of the variable j, and a moving image of t1 + t2 + t3 + t4 seconds is played for each article.

According to the present embodiment, the reproduction time determination processing unit 30 observes the line of sight of the observer using the eye tracking camera 40, and determines how much time is required for monitoring for each observer. Can be determined and memorized.

Further, since the monitoring processing unit 32 monitors while playing the moving image based on the personal playing time stored in advance, the moving image playing time can be set to a necessary and sufficient time, and the work efficiency can be improved. it can.

As a comparative example, if monitoring is performed for a fixed time (for example, the default value of 10 seconds), when the observer monitors the article 14 with a plurality of moving images, the waiting time becomes long and the work efficiency decreases. Become.

On the other hand, in the present embodiment, since the appropriate reproduction time tk is determined for each observer, the waiting time can be reduced and the article 14 can be monitored in a necessary and sufficient time.

In the present embodiment, the eye tracking camera 40 is arranged in order to obtain the line-of-sight information of the observer, and the movement of the line of sight is automatically detected. However, the eye tracking camera 40 is not limited to the eye tracking camera 40. A person's input operation (for example, key operation, button operation, etc.), utterance, or the like may be used as a trigger to indicate that the line of sight has moved.

Further, in the present embodiment, the reproduction time determination processing unit 30 generates an optimum synchronization difference to improve the monitoring work efficiency in the monitoring processing unit 32. However, the present invention is related to, for example, video distribution. It is also applicable to zapping of multiple contents.

Further, in the present embodiment, the video processing unit 28, the playback time determination processing unit 30, and the monitoring processing unit 32 are collectively managed by a processing program by the CPU 22A of the main control unit 22, but each processing unit is independent. It may be operated by the processing program of the CPU. Further, any one CPU may control the other processing unit.

Further, structurally, each may have an individual device configuration (the video processing unit 28 is unitized as a video processing device, the reproduction time determination processing unit 30 is unitized as a reproduction time determination processing device, and the monitoring processing unit 32 is a monitoring processing device). .. In this case, the reproduction time determination processing mode and the monitoring processing mode may be constructed by combining them according to the intended use and exchanging information through a communication interface or the like.

In the present embodiment, the function realized by software when the CPU executes a program may be realized by hardware such as a semiconductor integrated circuit, or by a combination of hardware and software. You may.

Further, in the present embodiment, the mode in which the processing program is stored (installed) in advance has been described, but the present invention is not limited to this. The processing program may be provided in a form recorded on a recording medium such as a CD-ROM (Compact Disc Read Only Memory), a DVD-ROM (Digital Versatile Disc Read Only Memory), and a USB (Universal Serial Bus) memory. .. Further, the processing program may be downloaded from an external device via a network.

10 Goods transport line 12 Main road 12A Belt conveyor 14 Goods 16A, 16B Branch road 18 Surveillance camera 18G Surveillance camera group 20 Surveillance system 22 Main control unit 22A CPU
22B RAM
22C ROM
22D input / output unit (I / O)
22E bus 24 User interface 28 Video processing unit 30 Playback time determination processing unit (timing information setting unit)
32 Monitoring processing unit (display control unit)
26 Large-scale storage device 26T Individual table 34 Video information storage unit 36 Monitor 38 Playback processing unit 40 Eye tracking camera 42 Execution management unit 44 Synchronous difference default value reading unit 46 Line-of-sight extraction unit 48 Synchronous difference default value storage unit 50 Playback time determination Unit 52 1st counter 54 Video playback time table generation unit 56 Monitoring mode execution instruction reception unit 58 Personal identification information acquisition unit 60 Video playback time table reading unit 62 Video playback time table acquisition unit 64 Table temporary storage unit 66 2nd counter

Claims (6)

A processing device that visually monitors an image simultaneously reproduced and displayed on a display device based on a plurality of types of image information captured by a plurality of imaging devices in a predetermined imaging area while the observer moves the line of sight.
A display control unit that controls the display of each image based on the plurality of types of image information by shifting the playback display start timing on the display device based on the timing information associated with each of the plurality of types of shooting information. Processing equipment to have. A specific object is being transported along the transport device,
The plurality of photographing devices are arranged at intervals along the transfer device, and the plurality of photographing devices are arranged at intervals.
The processing device according to claim 1, wherein each of the plurality of photographing devices photographs the specific object conveyed by the conveying device. The first or second aspect of claim 1 or 2, further comprising a timing information setting unit that associates the time difference information when the observer sequentially monitors the reproduced display video based on the plurality of video information as the timing information with each video information. Processing equipment. The eye tracking camera for detecting the line-of-sight information including the movement of the line-of-sight of the observer is provided.
As the timing information, time difference information extracted from the line-of-sight direction that the observer moves to confirm the reproduced display video based on the plurality of video information in a predetermined order based on the line-of-sight information is linked to each video information. The processing device according to claim 1 or 2, further comprising a timing information setting unit to be attached. A computer that visually monitors a video displayed simultaneously on a display device based on a plurality of types of video information shot by a plurality of shooting devices in a predetermined shooting area while the observer moves his / her line of sight.
As a display control unit that controls the display of each video based on the plurality of types of video information by shifting the playback display start timing on the display device based on the timing information associated with each of the plurality of types of shooting information. A processing program to make it work. This is a processing method in which a viewer visually monitors a predetermined shooting area while moving his / her line of sight, based on a plurality of types of video information shot by a plurality of shooting devices and simultaneously reproduced and displayed on a display device.
A processing method for displaying and controlling each video based on the plurality of types of video information by shifting the playback display start timing on the display device based on the timing information associated with each of the plurality of types of shooting information.