JP2022038374A - Information processing device, information processing method, and program - Google Patents

Abstract

To provide a technique for easily grasping the frame interval of an image in which analysis processing is executed.SOLUTION: An information processing device includes setting means that sets a frame interval in which a predetermined analysis process is executed for a moving image captured by imaging means, acquisition means that acquires the analysis result of a predetermined analysis process executed on the moving image according to a frame interval set by the setting means, and display control means that causes display means to display an index indicating an imaging period of the moving image corresponding to the analysis result among the moving images captured by the imaging means, and the display control means changes the display mode of the index according to the frame interval in which the predetermined analysis process is executed on the moving image corresponding to the analysis result.SELECTED DRAWING: Figure 6

Description

The present invention relates to information processing technology.

Conventionally, image analysis processing such as detection processing for detecting an object from an image and counting processing for counting the number of objects contained in the image is known. Further, there is known a technique of accumulating the result of analysis processing for an image in association with the imaging date and time of the image and displaying information indicating the result of analysis processing on a timeline.

Patent Document 1 discloses a description that information on a period during which a person is detected from an image is displayed on a timeline.

Japanese Unexamined Patent Publication No. 2019-125911

However, when the result of a predetermined analysis process is displayed on the timeline using the technique described in Patent Document 1, the user cannot confirm the frame interval of the image in which the analysis process is executed, so that the user can perform the analysis process. It may not be possible to know if it has been fully executed.

Therefore, an object of the present invention is to provide a technique for easily grasping the frame interval of an image in which an analysis process is executed.

In order to solve the above problems, the information processing apparatus of the present invention has, for example, the following configuration. That is, a setting means for setting a frame interval in which a predetermined analysis process is executed for the moving image captured by the imaging means, and a coin executed for the moving image according to the frame interval set by the setting means. An acquisition means for acquiring an analysis result of a predetermined analysis process, and a display control means for displaying an index indicating an imaging period of the moving image corresponding to the analysis result among the moving images captured by the imaging means on the display means. On the display screen, the display control means changes the display mode of the index according to the frame interval in which the predetermined analysis process is executed on the moving image corresponding to the analysis result.

According to the present invention, it is possible to easily grasp the frame interval of the image in which the analysis process is executed.

It is a figure which showed an example of the system configuration. It is a figure which shows an example of a functional block. It is a figure which shows an example of the appearance that a predetermined analysis process was executed with respect to an image. It is a figure which shows an example of the setting information to be recorded. It is a figure which shows an example of the analysis result information recorded. This is an example of a display screen displayed on the display 130. It is a flowchart which shows the flow of the process which accumulates the result of the predetermined analysis process for an image. It is a flowchart which shows the flow of the process which displays the display screen on a display 130. This is an example of a display screen displayed on the display 130. This is an example of a display screen displayed on the display 130. This is an example of a display screen displayed on the display 130. This is an example of a display screen displayed on the display 130. It is a figure which shows the hardware configuration of each apparatus.

Hereinafter, embodiments according to the present invention will be described with reference to the accompanying drawings. The configuration shown in the following embodiments is merely an example, and is not limited to the configurations shown in the drawings.

(Embodiment 1)
FIG. 1 is a diagram showing a system configuration in this embodiment. The system in this embodiment includes an information processing device 100, an image pickup device 110, a recording device 120, and a display 130.

The information processing device 100, the image pickup device 110, and the recording device 120 are connected to each other via the network 140. The network 140 is realized from a plurality of routers, switches, cables and the like conforming to a communication standard such as ETHERNET (registered trademark).

The network 140 may be realized by the Internet, a wired LAN (Local Area Network), a wireless LAN (Wireless Lan), a WAN (Wide Area Network), or the like.

The information processing apparatus 100 is realized by, for example, a personal computer or the like in which a program for realizing the information processing function described later is installed.

The image pickup device 110 is a device for taking an image and functions as an image pickup means. In the present embodiment, the case where a plurality of image pickup devices 110 are connected to the information processing device 100 and the recording device 120 via the network 140 will be described, but the image pickup device 110 may be one. Each of the plurality of image pickup devices 110 associates the image data of the captured image, the information of the capture date and time when the image was captured, and the identification information for identifying the image pickup device 110 that captured the image, and provides information via the network 140. It is transmitted to the processing device 100, the recording device 120, and the like. The information processing device 100 and the recording device 120 use, for example, the identification information associated with the transmitted image to determine which image pickup device 110 among the plurality of image pickup devices 110 has captured the transmitted image. do.

The recording device 120 records the image data of the image captured by the image pickup device 110, the information on the date and time when the image was captured, and the identification information for identifying the image pickup device 110 in association with each other. Then, in accordance with the request from the information processing device 100, the recording device 120 transmits the recorded data (image, identification information, etc.) to the information processing device 100.

The display 130 is composed of an LCD (Liquid Crystal Display) or the like, and displays the information processing result of the information processing apparatus 100, the image captured by the imaging apparatus 110, and the like. The display 130 is connected to the information processing apparatus 100 via a display cable compliant with a communication standard such as HDMI (registered trademark) (High Definition Multimedia Interface). At least two or all of the display 130, the information processing device 100, and the recording device 120 may be provided in a single housing.

The result of information processing by the information processing device 100 and the image captured by the image pickup device 110 are not limited to the display 130 connected to the information processing device 100 via the display cable, for example, the following external device. It may be displayed on the display of. That is, it may be displayed on a display of a mobile device such as a smartphone or a tablet terminal connected via the network 140.

Next, the information processing of the information processing apparatus 100 according to the present embodiment will be described with reference to the functional block of the information processing apparatus 100 according to the present embodiment shown in FIG.

In the case of this embodiment, each function shown in FIG. 2 is realized as follows by using a ROM (Read Only Memory) 1320 and a CPU (Central Processing Unit) 1300, which will be described later with reference to FIG. It shall be done. That is, each function shown in FIG. 2 is realized by the CPU 1300 of the information processing apparatus 100 executing the computer program stored in the ROM 1320 of the information processing apparatus 100.

The acquisition unit 200 sequentially acquires images of each frame constituting the moving image captured by the image pickup apparatus 110. The acquisition unit 200 may acquire the moving image transmitted from the imaging device 110, or may acquire the moving image transmitted from the recording device 120.

The recording unit 201 can be realized by a RAM (Random Access Memory) 1310, an HDD (Hard Disk Drive) 1330, or the like, which will be described later with reference to FIG. 13, and records information and data related to information processing by the information processing apparatus 100.

The operation receiving unit 202 receives an operation performed by the user via an input device (not shown) such as a keyboard or a mouse. For example, the operation receiving unit 202 accepts a user operation for designating a frame interval in which a predetermined analysis process is executed for a moving image.

The setting unit 203 sets a frame interval in which a predetermined analysis process is executed for the moving image acquired by the acquisition unit 200. For example, when the operation receiving unit 202 accepts a user operation for specifying a frame interval in which a predetermined analysis process is executed, the setting unit 203 determines a specified frame interval for a moving image based on the user operation. Set as the frame interval at which the analysis process of is executed. The information of the frame interval set by the setting unit 203 is recorded by the recording unit 201. The frame interval setting process by the setting unit 203 will be described later.

The image analysis unit 204 executes a predetermined analysis process on the moving image acquired by the acquisition unit 200 according to the frame interval set by the setting unit 203. In the present embodiment, the predetermined analysis process is related to the detection process of detecting a specific object from the image included in the moving image. For example, the predetermined analysis process includes a detection process for detecting a specific object for an image of a frame included in a moving image, a counting process for counting the number of specific objects detected by the detection process in the image, and the like. In the detection process for detecting a specific object from an image, for example, a pattern matching process or the like may be used, or a process for detecting a specific object from an image using a classifier learned by machine learning may be used. May be good. Further, the predetermined analysis process may be a tracking process for tracking a specific object detected by the detection process in the image of the frame included in the moving image. The recording unit 201 acquires (holds) information on a predetermined analysis process executed by the image analysis unit 204 on the moving image according to the set frame interval. At this time, the recording unit 201 records information with the analysis result of the predetermined analysis process executed on the image.

The generation unit 205 generates a display screen including an index indicating the imaging period of the moving image corresponding to the analysis result among the moving images captured by the imaging device 110. At this time, the generation unit 205 changes the display mode of the index indicating the imaging period of the moving image according to the frame interval in which the predetermined analysis processing is executed for the moving image corresponding to the analysis result. The display control unit 206 causes the display 130 to display the display screen generated by the generation unit 205. Further, the display control unit 206 causes the display 130 to display the moving image captured by the image pickup device 110.

Here, the information processing of the information processing apparatus 100 in the present embodiment will be described in more detail with reference to FIGS. 3 to 8. FIG. 3 is a diagram showing an example of a state in which a predetermined image process is executed by the image analysis unit 204 on the image. In the following description, the image analysis unit 204 executes a detection process for detecting a person as a specific object included in the image, and executes a counting process for counting the number of persons detected by the detection process from the image. It shall be. As shown in FIG. 3, the image analysis unit 204 in the present embodiment executes a detection process for detecting a person included in the captured image 300, and further counts the persons detected from the image 300 by the detection process. Execute the counting process. Further, as shown in FIG. 3, the generation unit 205 superimposes the frame 301 on the image 300 at the position of the person detected from the image 300, and displays the label 302 including the number of people counted in the image 300 on the image 300. The output image superimposed on the may be generated. At this time, the display control unit 206 may display the output image generated by the generation unit 205 on the display 130. In the example shown in FIG. 3, assuming that the entire region of the image 300 is the target of the detection process and the counting process, 15 people are detected in the image 300, and the analysis result of "15" is obtained. There is. Therefore, the generation unit 205 superimposes the label 302 including the numerical value “15” on the image 300 as the counting result obtained by the counting process for the captured image 300.

Next, the information held by the recording unit 201 in the present embodiment will be described with reference to FIGS. 4 and 5. The setting information 400 shown in FIG. 4 is information recorded by the recording unit 201, and the following information is recorded. That is, in the setting information 400, the identification information 401 of the image pickup device 110, the analysis period 402 which is the period during which the predetermined analysis process is executed among the moving images captured by the image pickup device 110, and the frame interval at which the analysis process is executed. 403, is associated and recorded. In the example of FIG. 4, among the plurality of image pickup devices 110, the identification information 401 is recorded in the setting information 400 for the three image pickup devices 110 of “cameraA”, “camaraB”, and “cameraC”. For example, as shown in FIG. 4, among the moving images captured by the image pickup device 110 of the “camera A”, the moving images captured during the analysis period 402 “2020/7/30, 13: 00-14: 00” , The analysis process is executed every "3 frames" indicated by the frame interval 403. Further, in the example of the setting information 400, among the moving images captured by the image pickup device 110 of the “camera A”, the moving images captured during the analysis period 402 “2020/7/30, 15: 00-17: 00” are compared. The analysis process is executed every "1 frame" indicated by the frame interval 403. The information included in the setting information 400 recorded by the recording unit 201 is set according to, for example, an operation by the user. For example, the operation reception unit 202 receives a user operation for specifying the following information prior to executing the analysis process for the real-time live video. That is, the operation reception unit 202 provides information on "cameraA", information on the period during which the analysis process is executed "2020/7/30, 13: 00-14: 00", and information on the frame interval of "3 frames". Accepts the specified user operation. Then, the recording unit 201 has the identification information 401 “cameraA”, the analysis period 402 “2020/7/30, 13: 00-14: 00”, and the frame interval 403 “3” according to the information received by the operation reception unit 202. It is stored in the setting information 400 in association with the "frame".

Here, the analysis result information 500 shown in FIG. 5 will be described. The analysis result information 500 is information recorded by the recording unit 201, and is an analysis result obtained by executing a predetermined analysis process on the moving image captured by the image pickup apparatus 110 according to the setting information 400. Information is recorded. In the example shown in FIG. 5, the following information is recorded in the analysis result information 500. That is, the identification information 501 of the image pickup device 110, the frame number 502 of the image captured by the image pickup device 110, the image capture date and time 503 of the image, and the analysis result 504 of the analysis process for the image are recorded in association with each other. To. For example, in the analysis result information 500, the image is imaged by the image pickup device 110 of the identification information 501 “cameraA”, the frame number 502 is “0”, and the image pickup date and time 503 is “2020/7/30/13: 00”. The analysis result 504 "15" is stored. In the present embodiment, it is assumed that the predetermined analysis process executed on the moving image is a counting process of the number of specific objects included in the image. Therefore, the analysis result 504 shows each image. Information on the number of specific objects counted is stored.

In the present embodiment, the moving image corresponding to the analysis result among the moving images captured by the imaging device 110, in other words, the moving image in which the analysis result exists among the moving images captured by the imaging device is as follows. A moving image such as is shown. That is, it shows a moving image in which a predetermined analysis process is executed among the moving images captured by the image pickup apparatus 110. For example, it is assumed that the image pickup device 110 of "camera A" captures a moving image over a period of "2020/7/30, 13: 00-17: 00". At this time, in the example of FIGS. 4 to 5, among the moving images captured by the image pickup device 110 of "cameraA", the moving image in which the analysis result exists is "2020/7/30, 13: 00-14: 00". And "2020/7/30, 15: 00-17: 00" are shown. On the other hand, among the moving images captured by the image pickup apparatus 110 over the period of "2020/7/30, 13: 00-17: 00", the moving image for which no analysis result exists is, for example, "2020/7/30, The moving image taken in the period of "14:01 to 14:59" is shown.

Next, with reference to FIG. 6, the information generated by the generation unit 205 in the present embodiment and displayed on the display 130 by the display control unit 206 will be described. The display screen 600 shown in FIG. 6 shows a screen generated by the generation unit 205 and displayed on the display 130 by the display control unit 206. The generation unit 205 generates the display screen 600 in response to the acquisition of the analysis result information 500 by executing the analysis process according to the setting information 400, for example. For each of the image pickup devices 110, the generation unit 205 in the present embodiment generates a display screen 600 including an index indicating the imaging period of the moving image corresponding to the analysis result among the moving images captured by the image pickup device 110. Specifically, the generation unit 205 has an index 602a and an index indicating the imaging period of the moving image from which the analysis result is obtained on the timeline 601a, which is a figure corresponding to the time axis, for the image pickup device 110 of the “cameraA”. 603a is drawn. At this time, the generation unit 205 refers to the setting information 400 or the analysis result information 500, and the moving image captured by the image pickup device 110 of the “camera A” during the period of “2020/7/30, 13: 00-14: 00”. It is determined that the analysis process has been executed for the image. Then, the generation unit 205 sets the period of "2020/7/30, 13: 00-14: 00" as the imaging period of the moving image for which the analysis result is obtained, and sets the index 602a indicating the period on the timeline 601a. Draw at the position corresponding to the period. At this time, as shown in FIG. 6, the generation unit 205 is positioned on the timeline 601a corresponding to the period of “2020/7/30, 13:00 to 14:00”, and is 13: on the timeline 601a. An index 602a indicating the period is drawn at positions from 00 to 14:00. Similarly, the generation unit 205 referred to the setting information 400 or the analysis result information 500, and was imaged by the image pickup device 110 of the “camera A” during the period of “2020/7/30, 15:00 to 17:00”. It is determined that the analysis process has been executed for the moving image. At this time, the period of "2020/7/30, 15:00 to 17:00" is set as the imaging period of the moving image for which the analysis result is obtained, and the index 603a indicating the period is drawn on the timeline 601a. As described above, in the present embodiment, the generation unit 205 marks a certain image pickup device 110 as an index indicating the image pickup period of the moving image for which the analysis result is obtained on the timeline corresponding to the image pickup device 110. draw. In FIG. 6, the generation unit 205 according to the present embodiment refers to the setting information 400 or the analysis result information 500 for each of the image pickup device 110 of the “camera B” and the image pickup device 110 of the “camera C”, and the analysis result is obtained. An index indicating the imaging period of the moving image is drawn. For example, in FIG. 6, the generation unit 205 sets the index 602b indicating the imaging period of the moving image for which the analysis result is obtained for the image pickup device 110 of the “camera B” on the timeline 601b corresponding to the image pickup device 110 of the “camera B”. It is drawn at the position corresponding to the image pickup period. Further, in FIG. 6, the generation unit 205 sets the index 602c indicating the imaging period of the moving image for which the analysis result is obtained for the image pickup device 110 of the “cameraC” on the timeline 601c corresponding to the image pickup device 110 of the “cameraC”. It is drawn at the position corresponding to the image pickup period.

Further, as shown in FIG. 6, the generation unit 205 is an index indicating the imaging period of the moving image corresponding to the analysis result according to the frame interval in which the analysis processing is executed for the moving image corresponding to the analysis result. The display mode of is different. For example, the generation unit 205 has a frame interval in which a predetermined analysis process is executed on the moving image captured by the image pickup device 110 of the "camera A" during the period of "2020/7/30, 13: 00-14: 00". Therefore, the display mode of the index 602a is changed. In the present embodiment, for example, the generation unit 205 makes the index color lighter as the frame interval in which the predetermined analysis process is executed is larger, and darkens the index color as the frame interval is smaller. In the example of the setting information 400 shown in FIG. 4, the frame interval corresponding to the moving image captured by the image pickup device 110 of the "camera A" during the image pickup period of "2020/7/30, 13: 00-14: 00" is ". 3 frames ". Further, in the example of the setting information 400, the frame interval corresponding to the moving image captured by the image pickup device 110 of the "camera A" during the image pickup period of "2020/7/30, 15:00 to 17:00" is "1 frame". Is. Therefore, as shown in FIG. 6, the generation unit 205 lightens the color of the index 602a corresponding to “2020/7/30, 13: 00-14: 00” on the timeline 601a, and “2020/7 / The color of the index 603a corresponding to "30, 13:00 to 17:00" is displayed darkly. In this way, the generation unit 205 displays an index indicating the imaging period of the moving image corresponding to the analysis result among the moving images captured by a certain imaging device 110, and the frame in which the analysis processing for the moving image is executed. The display mode of the index is changed according to the interval. In the present embodiment, the shade of the color of the index (index 602a, index 603a, etc.) is changed according to the frame interval, but the present invention is not limited to this. For example, the color of the index may be changed according to the frame interval, or the pattern of the pattern of the index may be changed. Then, the display control unit 206 causes the display 130 to display the display screen 600 generated by the generation unit 205.

Next, the information processing of the information processing apparatus 100 in the present embodiment will be described with reference to FIG. 7. By executing the process shown in FIG. 7, it is possible to execute a predetermined analysis process on the image and record the result of the analysis process. The process shown in FIG. 7 is executed by, for example, the functional block shown in FIG. 2 realized by executing the computer program stored in the ROM 1320 of the information processing apparatus 100 by the CPU 1300 of the information processing apparatus 100. ..

First, the information processing shown in FIG. 7 will be described. First, in S701, the acquisition unit 200 acquires an image of one frame as an image to be processed (hereinafter referred to as an image to be processed) among the images of a plurality of frames constituting the moving image captured by the image pickup apparatus 110. do. At this time, the acquisition unit 200 acquires the image to be processed, and also acquires the imaging date and time of the image to be processed and the identification information for identifying the image pickup device 110 that has imaged the image to be processed.

Next, in S702, the image analysis unit 204 determines whether the processing target image acquired in S701 is included in the analysis period. At this time, the image analysis unit 204, for example, refers to the setting information 400 set prior to the flow processing shown in FIG. 7, and executes the following processing. That is, the image analysis unit 204 determines whether the imaging date and time of the processing target image acquired in S701 is included in the analysis period 402 corresponding to the identification information 401 of the image pickup apparatus 110 that captured the processing target image. For example, when the image analysis unit 204 refers to the setting information 400 when the identification information associated with the image to be processed is “cameraA”, the image analysis unit 204 refers to the setting information 400 and performs the processing target during the analysis period 402 of “cameraA”. Determine if the image capture date and time is included. Then, when it is determined that the imaging date and time of the image to be processed acquired in S701 is not included in the analysis period 402 of the setting information 400 (No in S702), the transition to S701 is performed, and the acquisition unit 200 in S701 , Get the image of the next frame. On the other hand, when it is determined that the imaging date and time of the image to be processed acquired in S701 is included in the analysis period of the setting information 400 (Yes in S702), the process transitions to S703. The analysis period including the imaging date and time of the current image to be processed is defined as the target analysis period.

Then, in S703, the image analysis unit 204 determines whether or not the frame interval has elapsed from the frame in which the predetermined analysis process was executed last time to the frame of the current process target image, based on the frame interval corresponding to the target analysis period. judge. Here, a predetermined analysis process has already been executed for an image imaged by the image pickup device 110 of "cameraA", the image pickup date and time 503 is "2020/7/30/13: 00", and the frame number 502 is "0". It is assumed that the analysis result 504 is obtained. Here, the identification information corresponding to the current processing target image is "cameraA", the frame number indicating the frame order of the processing target image is "3", and the imaging date and time of the processing target image is "2020/7/30/13: 03". In the case of, the image analysis unit 204 makes the following determination in S703. That is, the image analysis unit 204 has a frame interval corresponding to the target analysis period from the image of the frame number “0” where the predetermined analysis process was executed last time to the current processing target image of the frame number “3” for “cameraA”. 403 It is determined that only "3 frames" have passed. At this time, since it becomes Yes in S703, the transition to S704 is performed. On the other hand, the identification information "cameraA" corresponding to the current processing target image, the frame number "1" indicating the frame order of the processing target image, and the imaging date and time "2020/7/30/13: 01" of the current processing target image. In this case, the image analysis unit 204 makes the following determination in S703. That is, the image analysis unit 204 has a frame interval "3 frames" corresponding to the target analysis period from the image having the frame number "0" on which the predetermined analysis process was executed last time to the current image to be processed having the frame number "1". It is determined that only has not passed. At this time, the result becomes No in S703, the transition to S701 occurs, and in S701, the acquisition unit 200 acquires an image to be processed next. If the current processed image is the first image obtained during the target analysis period, the transition to S704 is assumed to be Yes in S703. For example, the identification information corresponding to the current processing target is "cameraA", the frame number 502 indicating the frame order of the processing target image is "0", and the imaging date and time 503 of the current processing target image is "2020/7/30/13". : 00 ”is assumed. At this time, since the image to be processed is the first image acquired during the target analysis period "2020/7/30, 13:00 to 14:00", it is assumed that the transition to S704 is made.

In S704, the image analysis unit 204 executes a predetermined analysis process on the current image to be processed. The image analysis unit 204 in the present embodiment executes a detection process for detecting a specific object with respect to the current image to be processed, and further, a counting process for counting the specific object detected by the detection process from the image to be processed. To execute. Next, in S705, the recording unit 201 stores and records the analysis result of the predetermined analysis process for the current image to be processed in the analysis result information 500. In the present embodiment, a counting process is assumed as a predetermined analysis process. Therefore, the recording unit 201 includes identification information 501 corresponding to the image to be processed, a frame number 502 of the image to be processed, an imaging date and time 503 of the image to be processed, and an analysis result 504 (result of counting processing) for the image to be processed. Is associated and stored in the analysis result information 500.

When there is an instruction to end in S706 (Yes in S706), the process of the flow shown in FIG. 7 is terminated. On the other hand, if there is no end instruction in S706 (No in S706), the transition to S701 is performed, and in S701, the acquisition unit 200 acquires an image to be processed next.

Subsequently, the flow shown in FIG. 8 will be described. By executing the process shown in FIG. 8, the display 130 can display an index indicating the imaging period of the moving image corresponding to the analysis result among the moving images captured by the imaging device 110. The process shown in FIG. 8 is, for example, executed by the functional block shown in FIG. 2 realized by executing the computer program stored in the ROM 1320 of the information processing apparatus 100 by the CPU 1300 of the information processing apparatus 100. ..

First, in S801, the generation unit 205 acquires information regarding the analysis result of the predetermined analysis process executed on the moving image. Specifically, the generation unit 205 acquires, for example, the analysis result information 500 shown in FIG.

Next, in S802, the generation unit 205 generates a display screen for display on the display 130 based on the information acquired in S801. As shown in FIG. 6, the generation unit 205 in the present embodiment sets as an index indicating the imaging period of the moving image corresponding to the analysis result among the moving images captured by the imaging device 110 for each of the plurality of imaging devices. Generate the display screen displayed on the line. At this time, the generation unit 205 changes the display mode of the index according to the frame interval in which the analysis process for the moving image is executed.

Next, in S803, the display control unit 206 causes the display 130 to display the display screen generated by the generation unit 205 in S802. For example, the display control unit 206 causes the display 130 to display the display screen 600 generated by the generation unit 205.

In a system composed of a plurality of image pickup devices 110, the user may want to confirm the results of each of the plurality of image pickup devices 110 at the same time while comparing them. Therefore, when changing the temporal display range of the timeline on the displayed display screen, the timeline of the other image pickup apparatus 110 is changed according to the change of the display range of the timeline of the other image pickup apparatus 110. The display range may be changed. Here, for example, as shown in FIG. 9A, it is assumed that the display control unit 206 displays the display screen 700a generated by the generation unit 205 on the display 130. Further, the operation reception unit 205 drags the mouse 901 from the right to the left direction 902 on the timeline 601a as a user operation for changing the temporal display range of the timeline 601a corresponding to the image pickup device 110 of the “cameraA”. It is assumed that the operation is accepted. At this time, the display control unit 206 changes the first display range on the timeline to the second display range corresponding to the time zone after the time zone of the first display range according to the operation of dragging in the direction 902. .. For example, the display control unit 206 has the first display range “2020/7/30, 13: 00-17: 00” on the timeline to the second display range “2020 /” based on the operation of dragging the mouse 901 in the direction 902. Change to "7/30, 16: 00-20: 00". That is, the temporal display range of the timelines 601a to 601c corresponding to each of "cameraA" to "cameraC" is from "2020/7/30, 13:00 to 17:00" to "2020/7/30," It will be changed from 16:00 to 20:00. The display screen 900b shown in FIG. 9B is a display screen displayed on the display 130 by the display control unit 206 after the first display range is changed to the second display range according to a user operation. In this way, when changing the temporal display range of the timeline on the display screen, the display range of the timeline of the other image pickup apparatus 110 is changed according to the change of the display range of the timeline of one image pickup apparatus 110. You may change it.

Further, the generation unit 205 in the present embodiment may divide the plurality of image pickup devices 110 into a plurality of groups and generate a display screen for each of the plurality of groups. For example, in the example of FIG. 10, the generation unit 205 generates the display screen 1001 for the three image pickup devices 110 of the first group (group A), "cameraA" to "cameraC". Further, the generation unit 205 generates a display screen 1002 for the three image pickup devices 110 of the second group (group B), “cameraD” to “cameraF”. Then, the display control unit 206 displays the display screen 1001 corresponding to the first group and the display screen 1002 corresponding to the second group side by side on the display 130 at the same time. At this time, as shown in FIG. 10, the generation unit 205 may generate the display screen 1001 and the display screen 1002 so that the timeline is displayed in a different display range for each group. Specifically, the generation unit 205 is attached to each of the three image pickup devices 110 of "cameraA" to "cameraC" in the display range of "2020/07/30, 13:00 to 17:00" for the first group. Generate a display screen 1001 including the corresponding timeline. Then, the generation unit 205 has a time corresponding to each of the three image pickup devices 110 of "cameraD" to "cameraF" in the display range of "2020/07/29, 18:00 to 22:00" for the second group. Generate a display screen 1002 including lines. Further, when changing the temporal display range of the timeline on the display screen, another image pickup device belonging to the same group as the one image pickup device 110 is matched with the change of the timeline display range of one image pickup device 110. The display range of the 110 timeline may be changed. For example, when a user operation of dragging the mouse from right to left is performed on the timeline of "cameraA" of the first group, the display control unit 206 changes the display range of the timeline only for the first group. You may do so. That is, the display range can be changed separately for each group.

Further, the information processing apparatus 100 in the present embodiment displays an index indicating the imaging period of the moving image corresponding to the analysis result among the captured moving images, and the index is determined according to the frame interval in which the analysis processing is executed. However, the display mode is different, but the display mode is not limited to this. For example, the information processing apparatus 100 displays an index indicating the imaging period of the moving image corresponding to the analysis result among the captured moving images, and displays the index according to the result of the analysis processing for the moving image. It may be different. For example, it is assumed that a counting process for the number of specific objects included in an image is executed as a predetermined analysis process. At this time, for example, in the example of FIG. 6, the generation unit 205 draws the index 602a on the timeline 601 corresponding to “cameraA” and the period “2020/7/30, 13: 00-14” indicated by the index. The display mode is changed according to the counting result at ": 00". Specifically, the color of the index 602a may be changed according to the change in the counting result in "2020/7/30, 13: 00-14: 00". For example, the average of the counting results in "2020/7/30, 13: 00-13: 30" is "100", and the average of the counting results in "2020/7/30, 13: 30-13: 40" is ". It is assumed that the value is "30". At this time, in the index 602a, the color of the portion corresponding to "2020/7/30, 13: 00-13: 30" is displayed darkly, and 2020/7/30, 13: The color of the counting result portion at "30 to 13:40" may be displayed lightly. By doing so, the user can easily grasp the result of the counting process on the timeline.

As described above, the information processing apparatus 100 in the present embodiment causes the display 130 to display an index indicating the imaging period of the moving image corresponding to the analysis result among the moving images captured by the imaging device 110. At this time, the information processing apparatus 100 changes the display mode of the index according to the frame interval in which the predetermined analysis process is executed for the moving image. By doing so, it is possible to grasp the imaging period in which the analysis process is executed, and it is possible for the user to easily grasp the frame interval of the image in which the analysis process is executed.

(Embodiment 2)
The following can be considered when a predetermined analysis process is executed in real time on a moving image captured in real time by the image pickup apparatus 110. That is, if the number of image pickup devices 110 is large, the amount of data of the moving image in which the analysis process is executed increases, so that the frame interval in which the predetermined analysis process is executed may be increased in order to reduce the processing load. .. On the other hand, if the number of image pickup devices 110 is small, the amount of data of the moving image on which the revision processing is executed decreases, so that the frame interval in which the predetermined analysis processing is performed can be reduced. As described above, the frame interval in which the analysis process is executed may differ depending on the time zone depending on the situation in which the analysis process is executed. At this time, in order to obtain more sufficient analysis results later, there may be a use case in which the analysis process is to be executed again in a time zone in which the frame interval is large and the analysis process is not sufficiently performed. Therefore, the information processing apparatus 100 in the present embodiment corresponds to the designated time range based on the operation of designating the time range for which the analysis process is to be re-executed while the generated display screen is displayed. The predetermined analysis process is re-executed for the moving image. The parts different from those of the first embodiment will be mainly described, and the same components and the same reference numerals are given to the components and the processes which are the same as or the same as those of the first embodiment, and the duplicated description will be omitted.

FIG. 11 is a diagram for explaining information processing by the information processing apparatus 100 in the present embodiment. Here, in a situation where the display screen 1100a including the timelines 1101a to 1101c corresponding to each of the three image pickup devices 110 of "cameraA" to "cameraC" is displayed, the operation reception unit 202 performs the following user operation. Is assumed. That is, the operation receiving unit 202 drags the mouse 1102 from the upper left to the lower right in the direction 1103 as shown in FIG. 11A, and receives a user operation for designating the time range 1104 on the timeline 1101b. At this time, the generation unit 205 draws a time range 1104 whose vertices are each of the start point and the end point of the drag operation of the mouse 1102 on the display screen 1100a, and the display control unit 205 draws the time range 1104 on the display screen. The 1100a is displayed on the display 130. Furthermore, the operation receiving unit 202 accepts a user operation for changing the frame interval in which a predetermined analysis process is executed for the moving image corresponding to the time range 1104. For example, when the time range 1104 is specified, the display control unit 205 causes the display 130 to display numerical information of the current frame interval with respect to the time range 1104. At this time, when the operation reception unit 202 accepts the user operation for changing the displayed frame interval value, the setting unit 203 updates the frame interval with respect to the time range 1104 to the changed value. Here, for example, the current frame interval for the moving image of "2020/7/25, 18:00 to 19:00" indicated by the time range 1104 on the timeline 1101b is "10 frames", and the frame interval after the update is updated. Is assumed to be "3 frames". At this time, the image analysis unit 204 performs a predetermined analysis process executed on the moving image captured by the “camera B” at “2020/7/25, 18:00 to 19:00” at a frame interval of “3 frames”. The predetermined analysis process is re-executed for the moving image so as to be. For example, the image analysis unit 204 executes a predetermined analysis process every “3 frames” for the moving image captured by the “camera B” at “2020/7/25, 18:00 to 19:00”. At this time, for example, the recording unit 201 performs analysis processing performed every "10 frames" last time on the moving image captured by "cameraB" at "2020/7/25, 18:00 to 19:00". The result is deleted from the analysis result information 500. Then, the recording unit 201 stores the result of the analysis processing for each "3 frames" newly executed for the moving image in the analysis result information 500. After obtaining the analysis result of the analysis process newly executed according to the updated frame interval for the moving image captured in the imaging period of the time range 1104, the generation unit 205 displays the display screen on the display 130. Update. That is, the generation unit 205 changes the display mode of the index corresponding to the period of "2020/7/25, 18:00 to 19:00" of "cameraB" indicated by the imaging range 1104. In the display screen 1100b shown in FIG. 11B, the frame interval for the period of "2020/7/25, 18: 00-19: 00" of "cameraB" has been updated from "10 frames" to "3 frames". It is a screen updated by the generation unit 205 accordingly. As described above, the information processing apparatus 100 in the present embodiment displays a display screen including a timeline on which an index indicating the period in which the analysis result is obtained is drawn, and at the same time, within the time range designated on the timeline. Accepts user operations to update the corresponding frame interval. Then, the information processing apparatus 100 executes the analysis process again according to the updated frame interval. By doing so, it is possible to easily identify a time zone in which the frame interval is larger than in other time zones, and the analysis process is executed again so that the frame interval of the analysis process for the time zone becomes smaller. Instructions can be facilitated.

(Embodiment 3)
If the timeline has a wide temporal display range, it may be difficult for the user to find a time zone with a frame interval that deviates from the frame interval corresponding to other time zones. Therefore, in the information processing apparatus 100 of the present embodiment, when displaying the timeline, there is a place where the analysis interval is extremely long as compared with the surrounding analysis results within the range where the analysis results are continuously obtained. If so, you can place a marker at that location. The parts different from those of the first embodiment will be mainly described, and the same components and the same reference numerals are given to the components and the processes which are the same as or the same as those of the first embodiment, and the duplicated description will be omitted.

The display screen 1200 shown in FIG. 12 is an example of a display screen generated by the generation unit 205 in the present embodiment and displayed by the display control unit 206. The display screen 1200 includes a timeline 1201 indicating the presence or absence of the analysis result, and a marker 1203 indicating that the predetermined position includes the analysis result having a longer frame interval than the analysis result of the surrounding time zone. Further, the display screen 1200 includes a time range 1204 showing a time range to be displayed, and a timeline 1202 that enlarges and displays the time range 1204. The table control unit 206 in the present embodiment may display the timeline 1202 in response to a user operation in which the marker 1203 is designated by a mouse (not shown). It should be noted that the marker is displayed at a place where the frame interval is longer than the surrounding analysis result, but the condition for displaying the marker is not limited to this. For example, when performing image analysis by specifying a frame interval such as performing analysis processing on 10 frames per second, settings and actual settings such as where only one image could be analyzed per second. The marker 1203 may be displayed at a place where a difference occurs from the operation of. As described above, according to the information processing apparatus 100 in the present embodiment, the user can easily grasp the unexpected operation in the analysis by displaying the marker at the place where there is a difference from the surrounding analysis result on the timeline. Will be possible.

(Other embodiments)
Next, with reference to FIG. 13, the hardware configuration of the information processing apparatus 100 for realizing each function of each embodiment will be described. Although the hardware configuration of the information processing device 100 will be described in the following description, it is assumed that the recording device 120 and the image pickup device 110 are also realized by the same hardware configuration.

The information processing apparatus 100 in the present embodiment includes a CPU 1300, a RAM 1310, a ROM 1320, an HDD 1330, and an I / F 1340.

The CPU 1300 is a central processing unit that controls the information processing device 100 in an integrated manner. The RAM 1310 temporarily stores a computer program executed by the CPU 1300. The RAM 1310 also provides a work area used by the CPU 1300 to execute processing. Further, the RAM 1310 functions as, for example, a frame memory or a buffer memory.

The ROM 1320 stores a program or the like for the CPU 1300 to control the information processing apparatus 100. The HDD 1330 is a storage device for recording image data and the like.

The I / F 1310 communicates with an external device via the network 140 according to TCP / IP, HTTP, or the like.

In the description of each of the above-described embodiments, an example in which the CPU 1300 executes the processing will be described, but at least a part of the processing of the CPU 1300 may be performed by dedicated hardware. For example, the process of displaying a GUI (GRAPHICAL USER INTERDEFACE) or image data on the display 130 may be executed by a GPU (GRAPHICS PROCESSING UNIT). Further, the process of reading the program code from the ROM 1320 and expanding it to the RAM 1310 may be executed by a DMA (Direct Memory Access) that functions as a transfer device.

The present invention can also be realized by a process in which one or more processors read and execute a program that realizes one or more functions of the above-described embodiment. The program may be supplied to a system or device having a processor via a network or storage medium. The present invention can also be realized by a circuit (for example, an ASIC) that realizes one or more functions of the above-described embodiment. Further, each part of the information processing apparatus 100 may be realized by the hardware shown in FIG. 13 or by software.

In addition, another device may have one or more functions of the information processing device 100 according to each of the above-described embodiments. For example, the image pickup apparatus 110 may have one or more functions of the information processing apparatus 100 according to each embodiment. It should be noted that each of the above-described embodiments may be combined, and for example, the above-mentioned embodiments may be arbitrarily combined.

Although the present invention has been described above with the embodiments, the above embodiments are merely examples of embodiment of the present invention, and the technical scope of the present invention is limitedly interpreted by these. It's not a thing. That is, the present invention can be implemented in various forms within a range that does not deviate from the technical idea or its main features. For example, a combination of the respective embodiments is also included in the disclosure contents of the present specification.

100 Information processing device 200 Acquisition unit 201 Recording unit 202 Operation reception unit 203 Setting unit 204 Image analysis unit 205 Generation unit 206 Display control unit

Claims (10)

A setting means for setting a frame interval in which a predetermined analysis process is executed for a moving image captured by the imaging means, and a setting means.
An acquisition means for acquiring the analysis result of a predetermined analysis process executed on the moving image according to a frame interval set by the setting means, and an acquisition means.
A display control means for causing the display means to display an index indicating the imaging period of the moving image corresponding to the analysis result among the moving images captured by the imaging means.
The display control means is an information processing apparatus characterized in that the display mode of the index is changed according to the frame interval in which the predetermined analysis process is executed on the moving image corresponding to the analysis result. .. The display control means is
When the frame interval in which the predetermined analysis process is executed for the moving image corresponding to the analysis result is the first frame interval, the index is displayed in the first display mode.
The information processing apparatus according to claim 1, wherein when the frame interval is a second frame interval different from the first frame interval, the index is displayed in a second display mode different from the first display mode. The display control means displays an index indicating the imaging period in a timeline, which is a figure corresponding to the time axis, in correspondence with the imaging period of the moving image corresponding to the analysis result.
The display control means is
When the frame interval in which the predetermined analysis process is executed for the moving image corresponding to the analysis result is the first frame interval, the index of the first display mode is displayed on the timeline. ,
When the frame interval in which the predetermined analysis process is executed for the moving image corresponding to the analysis result is the second frame interval, the index of the second display mode is displayed on the timeline. The information processing apparatus according to claim 2. The index of the first display mode is a diagram in which lines are drawn side by side at intervals based on the first frame interval on the timeline.
The information processing apparatus according to claim 3, wherein the index of the second display aspect is a diagram in which lines are drawn side by side at intervals based on the second frame interval on the timeline. The index of the first display aspect is a mark of the first color, and is
The information processing apparatus according to claim 2 or 3, wherein the index of the second display aspect is a mark of a second color different from the first color. The information processing apparatus according to any one of claims 1 to 5, wherein the predetermined analysis process relates to a process of detecting a specific object in an image included in the moving image. 6. The image pickup period of the moving image corresponding to the analysis result is the period of the moving image in which the specific object is detected among the moving images captured by the imaging means. The information processing device described in. The information processing apparatus according to claim 6 or 7, wherein the specific object is a person. A setting process for setting a frame interval in which a predetermined analysis process is executed for a moving image captured by an imaging means, and a setting step.
An acquisition step of acquiring the analysis result of a predetermined analysis process executed on the moving image according to the frame interval set in the setting step, and an acquisition step.
It has a display control step of displaying an index indicating an imaging period of the moving image corresponding to the analysis result among the moving images captured by the imaging means on the display means.
In the display control step, an information processing method characterized in that the display mode of the index is changed according to the frame interval in which the predetermined analysis process is executed on the moving image corresponding to the analysis result. .. A program for making a computer function as each means of the information processing apparatus according to any one of claims 1 to 8.

Images