JP2024014343A - Information processing system, information processing device, information processing method, program - Google Patents

Abstract

The present invention provides an information processing system, an information processing device, an information processing method, and a program that enable a more intuitive and bird's-eye view of the movement status of moving objects, pedestrians, and the like. An information processing system that displays images shot by a camera on a user terminal via a server acquires the images shot by the camera, and acquires position information indicating a plurality of positions of the camera within a predetermined period. , generate a movement trajectory K composed of line segments connecting a plurality of positions of the camera based on the position information, obtain a map, and with the movement trajectory K superimposed on the map. , displaying the map and the video on the user terminal. [Selection diagram] Figure 7

Description

Application for application of Article 30, Paragraph 2 of the Patent Act has been filed Website publication date: November 9, 2021, Website address: https://safie. co. jp/news/1454/ Site publication date: November 4, 2021, publication site address: https://support. safie. link/hc/ja/articles/4408659252621-Safie-Pocket2-%E3%83%9E%E3%83%83%E3%83%97%E3%83%93%E3%83%A5%E3%83%BC %E3%82%A2%E3%83%BC%E3%81%AE%E4%BD%BF%E3%81%84%E6%96%B9 Site publication date: Posted on November 15, 2021 Site address: https://ssl4. eir-parts. net/doc/4375/tdnet/2049953/00. pdf Site publication date: December 23, 2021, publication site address: https://note. com/onuki06/n/nf31e1d705050

The present disclosure relates to an information processing system, an information processing device, and an information processing method. Furthermore, the present disclosure relates to a program for causing a computer to execute the information processing method.

Patent Document 1 discloses a drive recorder configured to record video data showing the surrounding environment of a running vehicle in order to analyze the causes of vehicle accidents and the like. A drive recorder mounted on a vehicle is connected to a camera configured to capture video data. Further, video data captured by the camera is stored in a portable memory card. By connecting the memory card to a user terminal (playback device) such as a computer, the video data stored on the memory card is transferred to the user terminal, and then the drive recorder video is played back on the display screen of the user terminal. Ru. In this way, the owner or manager of the vehicle (hereinafter referred to as a user) can check the video from the drive recorder installed in the vehicle on the user terminal.

Japanese Patent Application Publication No. 2010-130114

By the way, in the drive recorder disclosed in Patent Document 1, in order to check the video of the drive recorder on the user terminal, the user first removes the memory card from the drive recorder and then connects the memory card to the user terminal. There is a need to. In this way, it takes time and effort to check the video on the user terminal.

In addition, although users can understand the surrounding environment of a moving object such as a vehicle within a predetermined period of time through images from a drive recorder, it is not possible to understand the surrounding environment of a moving object such as a vehicle within a predetermined period of time through images from a drive recorder. It is not possible to grasp intuitively and from a bird's-eye view what something is. Furthermore, in situations where the user is unable to ascertain what kind of travel route on the map the video was taken along, the user may be unable to determine the overall movement status of a moving object such as a vehicle based on the video alone. It is difficult to grasp it intuitively and from a bird's-eye view. From the above viewpoint, there is room to further improve the usability of a video display screen on which a video showing the movement status of a mobile object is displayed.

The present disclosure aims to provide an information processing system, an information processing device, an information processing method, and a program that enable a more intuitive and bird's-eye view of the movement status of moving objects, pedestrians, and the like.

An information processing system according to one aspect of the present disclosure is a system that displays images captured by a camera on a user terminal via a server. The information processing system acquires an image taken by the camera, acquires position information indicating a plurality of positions of the camera within a predetermined period, and connects the multiple positions of the camera based on the position information. A movement trajectory composed of line segments is generated, a map is obtained, and the map and the video are displayed on the user terminal with the movement trajectory superimposed on the map.

According to the above configuration, the map and video are displayed on the user terminal with the movement trajectory superimposed on the map. In this way, by viewing the movement trajectory superimposed on the map, the user can intuitively and bird's-eye understand from which position on the map the video was captured. Furthermore, the user can intuitively understand along what travel route on the map the video was captured. Further, the user can easily check the camera image on a user terminal communicatively connected to the server via the communication network (ie, online). Therefore, it is possible to provide an information processing system that enables a more intuitive and bird's-eye view of the movement status of moving objects, pedestrians, etc. online.

Further, the information processing system selects a predetermined position on the movement trajectory in response to a user's input operation on the user terminal, and displays a video including an image frame associated with the selected predetermined position. You may.

According to the above configuration, by selecting a predetermined position on the movement trajectory, the user can easily check a video including a video frame associated with the selected predetermined position. In this way, the user can easily check the camera image at a desired position through the user terminal.

Further, the information processing system may display a pin placed on the movement trajectory, and display an image including an image frame associated with a position indicated by the pin.

According to the above configuration, the user can intuitively and intuitively know from which position on the map the currently displayed camera image was captured by looking at the position of the pin placed on the movement trajectory. It can be grasped from a bird's-eye view.

Further, the information processing system automatically moves the pin along the movement trajectory, and, by following the movement of the pin on the movement trajectory, creates an image frame associated with a position indicated by the pin. may be displayed.

According to the above configuration, the image frame associated with the position indicated by the pin is displayed following the movement of the pin on the movement trajectory. In this way, by looking at the pin moving along the movement trajectory, the user can intuitively and bird's-eye view the position on the map from which the currently displayed camera image was taken. can be grasped.

Further, the information processing system may determine the predetermined period according to a user's input operation on the user terminal.

According to the above configuration, after the user specifies a predetermined period, the position information of the camera during the specified predetermined period is acquired. Thereafter, the movement trajectory obtained based on the position information during the designated predetermined period is displayed on the map. In this way, the user can easily confirm the movement trajectory of the camera (in particular, the movement trajectory of the moving object that moves together with the camera) during a desired period.

Further, the information processing system may determine the predetermined period by specifying a time zone defined by two different times, a specific date, or a period defined by two different dates.

According to the above configuration, the user can specify a time zone defined by two different times, a specific date, or a period defined by two different dates to determine the movement trajectory of the camera (in particular, the camera movement trajectory) in a desired period. It is possible to easily check the movement locus of the moving body moving with the vehicle.

The information processing system also acquires a moving speed of the camera associated with a predetermined line segment on the movement trajectory, and determines a visual aspect of the predetermined line segment according to the moving speed. Good too.

According to the above configuration, the user can easily grasp the moving speed of the camera (or the moving object that moves together with the camera) by looking at the visual aspect of the predetermined line segment.

Further, the position information includes first position information indicating a first position, second position information indicating a second position different from the first position, and second position information acquired after the first position information. May include. The movement trajectory may include a line segment connecting the first position and the second position. The information processing system controls the camera associated with the line segment based on the length of the line segment and the time interval between the acquisition time of the first position information and the acquisition time of the second position information. may determine the speed of movement.

According to the above configuration, the camera associated with the first line segment is determined based on the length of the first line segment and the time interval between the acquisition time of the first position information and the acquisition time of the second position information. The speed of movement is determined. In this way, the moving speed of the camera can be calculated only from the camera position information.

Further, the location information includes first location information indicating a first location, and second location information indicating a second location different from the first location and obtained after the first location information. But that's fine. When the time interval between the acquisition time of the first position information and the acquisition time of the second position information is larger than a predetermined time interval, the line segment connecting the first position and the second position is It may not be included in the movement trajectory.

According to the above configuration, when the time interval between the acquisition time of the first position information and the acquisition time of the second position information is larger than the predetermined time interval, the line connecting the first position and the second position is minutes are not included in the movement trajectory of the camera (or moving object). In this way, it is possible to suitably prevent a situation in which the movement trajectory of the camera determined based on the position information differs from the actual movement trajectory of the camera. For example, when a vehicle (particularly a vehicle equipped with a camera) is traveling in a tunnel, position information cannot be obtained from GPS or the like. In such a situation, if the first position information is acquired before the vehicle enters the tunnel, and the second position information is acquired after the vehicle exits the tunnel, the first position and the second position may be It becomes clear that the connecting line segments do not match the actual route of the tunnel. In this way, by excluding the line segment connecting the first position and the second position from the movement trajectory, a movement trajectory that does not feel strange (in other words, a movement trajectory that is more realistic) is displayed on the map. becomes possible.

The information processing system also determines a first position on the movement trajectory in response to a user's input operation on the user terminal, and displays a first image frame associated with the first position on the map. You may.

According to the above configuration, the user can immediately grasp the surrounding environment at the first position by looking at the first image frame displayed on the map.

Further, the information processing system specifies the occurrence of a predetermined event based on the video and the location information, and also specifies a place where the predetermined event occurs, and based on the place where the predetermined event occurs. Information related to the predetermined event may be displayed on the map.

According to the above configuration, the user can immediately grasp at which position on the map a predetermined event (for example, a traffic accident, etc.) has occurred.

Further, the information processing system acquires a moving speed of the camera associated with a predetermined line segment on the moving trajectory, and based on a comparison result between the moving speed and a predetermined speed, The visual aspect of the line segment may be changed.

According to the above configuration, the user can easily grasp the information related to the moving speed of the camera associated with the predetermined line segment by looking at the visual aspect of the predetermined line segment.

Further, the information processing system may change the visual aspect of the predetermined line segment when the moving speed is higher than a speed limit.

According to the above configuration, by looking at the visual aspect of the predetermined line segment, the user can easily understand that the moving object was traveling at a speed exceeding the speed limit in the predetermined line segment.

The information processing system also acquires a moving speed of the camera associated with a first position on the moving trajectory, and adjusts the moving speed to a predetermined speed based on a comparison result between the moving speed and a predetermined speed. Relevant information may be displayed on the map.

According to the above configuration, the user can easily grasp the information related to the moving speed at the first position by viewing the information related to the moving speed of the camera.

Further, the information processing system may receive data indicating whether the camera is moving from the camera, and determine whether the camera is moving based on the data. If the information processing system determines that the camera is moving within a first period, the information processing system transmits the position information within the first period to the user terminal, and determines that the camera is moving within a second period. If it is determined that the user terminal has not done so, the location information within the second period may not be transmitted to the user terminal.

According to the above configuration, if the camera is moving within the first period, position information within the first period is transmitted to the user terminal. On the other hand, if the camera does not move within the second period, position information within the second period is not transmitted to the user terminal. In this way, position information during a period when the camera (or mobile object) is not moving is not sent to the user terminal, so even when the camera is not actually moving, the movement trajectory of the camera may be changed due to the accuracy of GPS. This can be suitably prevented from causing any disturbance.

Furthermore, the camera may be mounted on a moving body or worn on a person.

According to the above configuration, it is possible to provide an information processing system that enables a more intuitive and bird's-eye view of the movement status of a moving object or a person.

An information processing device according to one aspect of the present disclosure is communicatively connected to a camera and a user terminal via a communication network, and includes a processor and a memory that stores computer-readable instructions. When the computer readable instructions are executed by the processor, the information processing device receives an image taken by the camera, receives position information indicating a plurality of positions of the camera within a predetermined period, and transmitting information and the video to the user terminal; The map and the video are displayed on the user terminal with the movement trajectory of the camera superimposed on the map. The movement trajectory is generated based on the position information and is composed of line segments connecting a plurality of positions of the camera.

According to the above configuration, it is possible to provide an information processing device that enables a more intuitive and bird's-eye view of the movement status of moving objects, pedestrians, and the like.

An information processing method according to one aspect of the present disclosure is executed by a user terminal. The user terminal is communicatively connected to a server via a communication network. The server is communicatively connected to the camera via the communication network. The information processing method includes a step of receiving an image taken by the camera from the server, a step of receiving position information indicating a plurality of positions of the camera within a predetermined period from the server, and a step based on the position information. a step of generating a movement trajectory composed of line segments connecting a plurality of positions of the camera; a step of acquiring a map; and a step of generating the map and the video image with the movement trajectory superimposed on the map. displaying on the user terminal.

According to the above, it is possible to provide an information processing method that enables a more intuitive and bird's-eye view of the movement status of moving objects, pedestrians, and the like.

Further, a program for causing a user terminal to execute the above information processing method may be provided.

According to the present disclosure, it is possible to provide an information processing system, an information processing device, an information processing method, and a program that enable a more intuitive and bird's-eye view of the movement status of moving objects, pedestrians, and the like.

1 is a diagram illustrating an example of a configuration of an information processing system according to an embodiment of the present disclosure (hereinafter referred to as the present embodiment). It is a diagram showing an example of the hardware configuration of a camera. It is a diagram showing an example of the hardware configuration of a server. It is a diagram showing an example of the hardware configuration of a user terminal. FIG. 2 is a sequence diagram for explaining a series of processes executed by the information processing system according to the present embodiment. FIG. 2 is a diagram for explaining an example of camera position information and a video configuration. FIG. 2 is a diagram (part 1) showing an example of a video display screen in which a movement trajectory of a camera is displayed on a map. FIG. 7 is a diagram (Part 2) showing an example of a video display screen in which a movement trajectory of a camera is displayed on a map; 12 is a flowchart illustrating a process for displaying a movement trajectory during a specified period specified by a user. FIG. 3 is a diagram showing an example of a video display screen on which a calendar is displayed. FIG. 3 is a diagram schematically showing a part of a movement trajectory. FIG. 6 is a diagram illustrating a state in which the visual aspect of a portion of line segments forming a movement trajectory has been changed. FIG. 6 is a diagram showing a state in which information related to the moving speed of the camera is displayed. FIG. 3 is a diagram (part 3) showing an example of a video display screen in which a movement trajectory of a camera is displayed on a map; FIG. 7 is a diagram illustrating an example of a video display screen in which a reduced screen of video is displayed on a map. FIG. 3 is a diagram showing an example of a video display screen on which event information is displayed on a map. 12 is a flowchart for explaining a process of transmitting only position information to a user terminal during a period in which the camera is moving. FIG. 7 is a diagram showing a relationship between a period in which the camera is moving or a period in which the camera is not moving and camera position information. FIG. 3 is a schematic diagram showing an example of a movement trajectory generated based on both positional information during a period when the camera is moving and positional information during a period when the camera is not moving. FIG. 3 is a schematic diagram showing an example of a movement trajectory generated only based on position information during a period in which the camera is moving.

The information processing system 1 according to the present embodiment will be described below with reference to the drawings. FIG. 1 is a diagram showing an example of the configuration of an information processing system 1 according to the present embodiment. As shown in FIG. 1, the information processing system 1 includes a camera 2, a server 3, a user terminal 4, and an external server 5. These are connected to a communication network 6. For example, the camera 2 is communicatively connected to the server 3 via the communication network 6. Further, it is assumed that the camera 2 is mounted on a vehicle 8 (an example of a moving body) or attached to a part of the body of a passenger of the vehicle 8. The user terminal 4 is communicatively connected to the server 3 via the communication network 6 .

In the description of the present embodiment, the camera 2 is mounted on the vehicle 8 or attached to a part of the occupant's body, so the position and movement trajectory of the camera 2, which will be described later, may be different from the position of the vehicle 8 or the movement trajectory. Matches the movement trajectory. Further, the camera 2 may be mounted on a moving body other than the vehicle 8 such as a train, a ship, an airplane, or an unmanned aerial vehicle (drone), or may be mounted on a part of the body of a person or an animal (pet). Good too. Furthermore, the vehicle 8 is not limited to a four-wheeled vehicle, but includes a two-wheeled vehicle and a three-wheeled vehicle. In this way, the number of wheels attached to the vehicle 8 is not particularly limited.

Further, in this embodiment, for convenience of explanation, a single camera 2 is provided in the information processing system 1, but the number of cameras 2 provided in the information processing system 1 is not particularly limited. For example, a plurality of cameras 2 associated with the user U may be provided in the information processing system 1. In this case, each of the plurality of cameras 2 may be mounted on a corresponding one of the plurality of vehicles 8. Furthermore, in the case where multiple users access the server 3 through user terminals, each of the multiple users is associated with at least one of the multiple cameras 2. In other words, each of the plurality of users may conclude a contract related to the video providing service of at least one of the plurality of cameras 2 (for example, a subscription contract, etc.) or a contract related to the use of the server 3.

The communication network 6 is configured by, for example, at least one of a LAN (Local Area Network), a WAN (Wide Area Network), the Internet, and a wireless core network.

Next, the hardware configuration of the camera 2 will be described below with reference to FIG. The camera 2 is configured to capture an image showing the surrounding environment and to transmit the captured image to the server 3 via the communication network 6. As shown in FIG. 2, the camera 2 includes a control section 20, a storage device 21, a position information acquisition section 22, an RTC (Real Time Clock) 23, an inertial sensor 24, a communication section 25, and an input operation section. 26, an imaging section 27, an audio input section 28, and a PTZ mechanism 29. These elements are connected to a communication bus 39. Further, the camera 2 may have a built-in battery (not shown).

The control unit 20 includes a memory and a processor. The memory is configured to store computer readable instructions (programs). For example, the memory includes a ROM (Read Only Memory) in which various programs are stored, a RAM (Random Access Memory) having a plurality of work areas in which various programs executed by a processor are stored, and the like. The processor includes, for example, at least one of a CPU (Central Processing Unit), an MPU (Micro Processing Unit), and a GPU (Graphics Processing Unit). The CPU may be composed of multiple CPU cores. A GPU may be configured by multiple GPU cores. The processor may be configured to load a designated program from various programs incorporated in the storage device 21 or the ROM onto the RAM, and execute various processes in cooperation with the RAM.

The storage device 21 is, for example, a storage device such as an HDD (Hard Disk Drive), an SSD (Solid State Drive), or a flash memory, and is used to store programs and various data (video data, location information, etc.). It is composed of The position information acquisition unit 22 is configured to acquire information regarding the current position (longitude θ, latitude φ) of the camera 2, and is, for example, a GPS (Global Positioning System) receiver. For example, the position information acquisition unit 22 may acquire information regarding the current position of the camera 2 at predetermined time intervals (for example, 10 seconds). Note that when the camera 2 is present in a mobile body, a situation is assumed in which the camera 2 is temporarily not connected to the communication network 6 depending on the circumstances of the surrounding environment of the mobile body (particularly, the state of wireless communication). Even in such a situation, since video data, position information, etc. are stored in the storage device 21, when the camera 2 is reconnected to the communication network 6, these data are not transmitted from the camera 2 to the server 3. It's okay. In this way, even when the mobile object is moving inside the tunnel, video data showing the situation inside the tunnel can be transmitted to the server 3 via the communication network 6.

The RTC 23 is configured to acquire information indicating the current time. In this regard, the control unit 20 acquires the position information of the camera 2 based on the information indicating the current time acquired by the RTC 23 and the information indicating the current position of the camera 2 acquired by the position information acquisition unit 22. do. In this respect, as shown in FIG. 6, the position information of the camera 2 includes information indicating the position Xn (longitude θn, latitude φn) of the camera 2 and information indicating the time tn at which the position of the camera 2 was acquired. (n is an integer of 1 or more). The position Xn of the camera 2 and the time tn at which the position of the camera 2 was acquired are associated with each other. For example, in the position information of the camera 2, the position X1 (latitude θ1, longitude φ1) of the camera 2 and the time t1 at which the position X1 was acquired are associated with each other. Similarly, the position X2 (latitude θ2, longitude φ2) of the camera 2 and the time t2 at which the position X2 was acquired are associated with each other. In this way, the position information of the camera 2 is composed of information regarding the plurality of positions Xn of the camera 2 and information regarding the plurality of times tn, each of which is associated with a corresponding one of the plurality of positions Xn. Note that instead of RTC, information indicating the current time may be acquired using NTP.

The inertial sensor 24 is configured to acquire data (eg, acceleration data, etc.) indicating whether the camera 2 (more specifically, the vehicle 8 on which the camera 2 is mounted) is moving. The inertial sensor 24 may be, for example, an acceleration sensor configured to acquire acceleration data indicating the acceleration of the camera 2 (vehicle 8).

The communication unit 25 is configured to connect the camera 2 to the communication network 6. The communication unit 25 includes, for example, a wireless communication module for wirelessly communicating with external devices such as a base station and a wireless LAN router. The wireless communication module includes a transmitting/receiving antenna and a signal processing circuit. The wireless communication module may be a wireless communication module compatible with short-range wireless communication standards such as Wi-Fi (registered trademark) or Bluetooth (registered trademark), or may be an X generation wireless communication module that uses a SIM (Subscriber Identity Module). It may be a wireless communication module compatible with a mobile communication system (for example, a fourth generation mobile communication system such as LTE).

The input operation unit 26 is configured to receive an input operation from an operator and to generate an operation signal according to the input operation from the operator. The imaging unit 27 is configured to capture an image showing the surrounding environment of the camera 2. In particular, the imaging unit 27 is configured to generate a video signal indicating the surrounding environment of the camera 2, and includes an optical system, an image sensor, and an analog processing circuit. The optical system includes, for example, an optical lens and a color filter. The image sensor is configured with a CCD (Charge-Coupled Device), a CMOS (Complementary MOS), or the like. The analog processing circuit is configured to process a video signal (analog signal) photoelectrically converted by the image sensor, and includes, for example, an amplifier and an AD converter. The control unit 20 generates video data (an example of image data) based on the video signal (digital signal) transmitted from the imaging unit 27. The frame rate of video data is, for example, 30 fps.

As shown in FIG. 6, in this embodiment, the video (video data) of the camera 2 is composed of image information composed of a plurality of image frames (still images), audio information, and time information. . The control unit 20 generates image information and audio information based on the video signal acquired from the imaging unit 27, the audio signal acquired from the audio input unit 28, and the information indicating the current time acquired from the RTC 23. , and time information (video data). Furthermore, each of the plurality of image frames is associated with the time at which the image frame was acquired. For example, image frame F1 is associated with the time at which image frame F1 was acquired. Similarly, image frame F2 is associated with the time at which image frame F2 was captured.

The audio input unit 28 is configured to generate an audio signal representing audio surrounding the camera 2. The audio input unit 28 is, for example, a microphone.

The PTZ mechanism 29 includes a pan (Panoramac) mechanism, a tilt (Tilt) mechanism, and a zoom (Zoom) mechanism. The panning mechanism is configured to change the orientation of the camera 2 in the horizontal direction. The tilt mechanism is configured to change the orientation of the camera 2 in the vertical direction. The zoom mechanism is configured to enlarge (zoom in) or reduce (zoom out) an image showing an object to be imaged by changing the angle of view of camera 2 . The zoom mechanism may change the angle of view of the camera 2 optically by changing the focal length of the optical lens included in the imaging unit 27, or may change the angle of view of the camera 2 digitally. . In this embodiment, in response to the user U's input operation on the user terminal 4, an instruction signal for instructing the camera 2 to pan, tilt, and/or tilt is transmitted from the user terminal 4 to the camera 2 via the server 3. In this case, the control unit 20 realizes the pan, tilt, and zoom functions (PTZ functions) of the camera 2 in real time by driving the PTZ mechanism 29 according to the received instruction signal. In this way, the PTZ function of the camera 2 can be realized through remote control of the user terminal 4. Note that the camera 2 does not need to include the PTZ mechanism 29.

The camera 2 can transmit a video (video data stream) showing the surrounding environment of the camera 2 to the server 3 in real time via the communication network 6 . Further, the camera 2 can transmit the position information of the camera 2, the orientation information of the camera 2, and data (acceleration data, etc.) indicating the movement of the camera 2 to the server 3 via the communication network 6.

Note that in this embodiment, the camera 2 may be indirectly connected to the communication network 6. For example, the communication unit 25 is communicably connected to the communication module of the vehicle 8 by short-range wireless communication such as Wi-Fi, while the communication module of the vehicle 8 is directly connected to the communication network 6. good. In this way, the camera 2 may be indirectly connected to the communication network 6 via the communication module of the vehicle 8.

Next, the hardware configuration of the server 3 will be described below with reference to FIG. The server 3 functions as a management server that manages camera information tables related to each camera, video data, and user information tables related to each user. The camera information table may include each camera's network address (eg, IP address), identification information, configuration information, and location information. The user information table may include attribute information and contract information of each user, and identification information of a camera associated with each user.

Furthermore, the server 3 also functions as a WEB server that transmits data (described later) for displaying a video display screen in response to a request from the user terminal 4. In this way, the server 3 functions as a server for providing SaaS (System as a Service), and may be composed of a plurality of servers. Further, the server 3 may be constructed on-premises or may be a cloud server.

The server 3 is communicably connected to the camera 2 and the user terminal 4 via the communication network 6, and is configured to transmit video data of the camera 2 in response to a request from the user terminal 4. The server 3 includes a control section 30, a storage device 31, a communication section 33, an input operation section 34, and a display section 35. These elements are connected to a communication bus 36.

The control unit 30 includes a memory and a processor. The memory is configured to store computer readable instructions (programs). For example, the memory is composed of ROM and RAM. The processor includes, for example, at least one of a CPU, an MPU, and a GPU.

The storage device 31 is, for example, a storage device such as an HDD, SSD, or flash memory, and is configured to store programs and various data. Various databases are constructed in the storage device 31. In particular, the storage device 31 may store camera information tables related to each camera, video data captured by each camera, and user information tables related to each user.

The communication unit 33 is configured to connect the server 3 to the communication network 6. Specifically, the communication unit 33 may include various wired connection terminals for communicating with external terminals on the communication network 6. The input operation section 34 is, for example, a touch panel, a mouse, a keyboard, etc. arranged over the display section 35, and receives input operations from an operator and also outputs operation signals according to the input operations from the operator. is configured to generate. The display section 35 is configured by, for example, a liquid crystal panel or an organic EL panel.

Next, the hardware configuration of the user terminal 4 will be described below with reference to FIG. The user terminal 4 may be, for example, a personal computer, a smartphone, a tablet, or a wearable device (for example, AR glasses or a head-mounted display) attached to the user U of the user terminal 4. The user terminal 4 may have a web browser for displaying data sent from the web server. The user terminal 4 includes a control section 40, a storage device 41, a communication section 43, an input operation section 44, and a display section 45. These elements are connected to a communication bus 46.

The control unit 40 includes a memory and a processor. The memory is configured to store computer readable instructions (programs). For example, the memory is composed of ROM and RAM. The processor includes, for example, at least one of a CPU, an MPU, and a GPU.

The storage device 41 is, for example, a storage device such as an HDD, SSD, or flash memory, and is configured to store programs and various data. The communication unit 43 is configured to connect the user terminal 4 to the communication network 6. The communication unit 43 includes a wireless communication module and/or a wired connection terminal for wirelessly communicating with a base station or a wireless LAN router. The input operation unit 44 is, for example, a touch panel, a mouse, and/or a keyboard arranged on the display unit 45, and receives input operations from the user U and outputs operation signals in response to the input operations from the user U. is configured to generate. The display section 45 is configured by, for example, a liquid crystal panel or an organic EL panel. In this embodiment, a video display screen 50 (see FIG. 7) including the video of the camera 2 and a map is displayed on the display unit 45.

(Series of processes executed by information processing system 1)
Next, a series of processes executed by the information processing system 1 according to this embodiment will be described below with reference to FIG. 5. In this example, camera 2 is associated with user U. In other words, the user U has concluded a contract regarding the video providing service of the camera 2 via the communication network 6. Note that each process of the server 3 is executed by the control unit 30 of the server 3. Similarly, each process of the user terminal 4 is executed by the control unit 40 of the user terminal 4.

As shown in FIG. 5, in step S1, the server 3 receives video (video data) from the camera 2 and position information and orientation information of the camera 2 through the communication network 6. Further, the server 3 stores the received video, position information, etc. in the storage device 31. Here, the camera 2 may continue to transmit the video data stream to the server 3 in real time, while transmitting position information to the server 3 at predetermined time intervals. Here, the orientation information of the camera 2 may include information indicating the orientation of the camera 2 in the horizontal direction.

Next, in step S2, the user terminal 4 transmits user U's login information (for example, login ID and login password) to the server 3 through user U's input operation. In step S3, the server 3 authenticates the user U based on the login information sent from the user terminal 4. Note that the authentication of the user U is not limited to login authentication, and various authentication methods may be applied. After authenticating the user U, the server 3 identifies the camera 2 associated with the user U by referring to the user information table.

In step S4, the server 3 transmits, via the communication network 6, data for displaying the video display screen 50 shown in FIG. The location information and direction information of are transmitted to the user terminal 4. Further, in this step, acceleration data indicating whether the camera 2 is moving may be transmitted from the camera 2 to the server 3.

The video display data may be a file (for example, an HTML file, a CSS file, a program file, etc.) for displaying the video display screen 50 on the web browser of the user terminal 4. Here, the program file may be a JavaScript file. The program file functions as a program file for causing the control unit 40 of the user terminal 4 to execute the processes of steps S5 and S6. Further, the program file may be stored in a computer-readable storage medium such as a flash memory. Note that the video display data (HTML file, etc.) may be transmitted only once. On the other hand, the video data of camera 2 may be transmitted in a streaming format.

In particular, the server 3 may transmit video data of a predetermined period out of the video data stored in the storage device 31 to the user terminal 4 in a streaming format. Similarly, the server 3 may transmit position information for a predetermined period out of the position information of the camera 2 stored in the storage device 31 to the user terminal 4. For example, the server 3 may transmit video data and position information from 0:00 to 24:00 on May 5, 2022 to the user terminal 4 as an initial state.

Next, in step S5, the user terminal 4 receives the position information of the camera 2 within a predetermined period transmitted from the server 3, and based on the position information, the user terminal 4 determines the movement trajectory K of the camera 2 (see FIG. 7). generate. The movement trajectory K is composed of line segments connecting multiple positions of the camera 2. For example, as shown in FIG. 11, if information indicating positions X1, X2, and X3 is included in the position information of camera 2, the movement trajectory K is a line segment L1 connecting positions X1 and X2, and It includes a line segment L2 connecting position X2 and position X3. The user terminal 4 executes the process of generating the movement trajectory K by executing the program file (computer-readable instructions) sent from the server 3. Furthermore, since the camera 2 moves together with the vehicle 8, the movement trajectory of the camera 2 coincides with the movement trajectory of the vehicle 8. In the following description, the movement trajectory K of the camera 2 may be referred to as the movement trajectory K of the vehicle 8.

In step S6, the user terminal 4 obtains a map (see FIG. 7) from the external server 5. In this regard, the user terminal 4 acquires a map covering the movement trajectory K of the camera 2 from the external server 5 based on the position information of the camera 2 through an API (Application Programming Interface) included in the video display data. . The map acquired from the external server 5 may be, for example, a Google map provided by Google LLC, headquartered in California, USA, or a map provided by another map providing company such as Zenrin Co., Ltd. It may be. Further, the map may be stored in advance in the storage device 41 of the user terminal 4. The type of map is not particularly limited to a map that includes roads, but may also be a map that shows the inside of private land such as a construction site, factory, farm, or ranch.

In step S7, the user terminal 4 (more specifically, the web browser operating on the user terminal 4) includes the video and the map based on the video display data, the video, the map, and the movement trajectory K. A video display screen 50 (see FIG. 7) is displayed on the display section 45. As shown in FIG. 7, the video display screen 50 is provided with a map display area 51, a video display area 52, and a timeline 53, respectively. The map is displayed in the map display area 51. The video is displayed in the video display area 52. The date and time displayed on the timeline 53 correspond to the date and time of the video (video frame) currently displayed in the video display area 52. Further, when the user U specifies a desired date and time on the timeline 53 by operating the slider 58, a video including a video frame corresponding to the specified date and time is displayed in the video display area 52.

Furthermore, in the map display area 51 of the video display screen 50, the movement trajectory K of the camera 2 is superimposed on the map. As described above, the movement trajectory K from 0:00 to 24:00 on May 5, 2022 is superimposed on the map as an initial state. A pin P is arranged on the movement trajectory K. The pin P specifies any one of a plurality of positions of the camera 2 forming the movement trajectory K. Further, a video including an image frame associated with the position indicated by the pin P is displayed in the video display area 52. Further, in the display area 63 visually associated with the pin P, information indicating the device name of the camera 2 and information regarding the time when the position indicated by the pin P was acquired are displayed.

In addition, by changing the position of the pin P on the movement trajectory K by a drag-and-drop operation, the user U displays a video including an image frame associated with the changed position of the pin P in the video display area 52. can be done.

As described above, the position information of the camera 2 is composed of information regarding a plurality of positions Xn of the camera 2 and information regarding a plurality of times tn, each of which is associated with a corresponding one of the plurality of positions Xn. . Therefore, for example, when the pin P indicates the position X2 of the camera 2, the time t2 associated with the position X2 (in other words, the time t2 when the position X2 was acquired by GPS etc.) is specified by the user terminal 4 be done. Thereafter, the user terminal 4 receives video data including the image frame associated with time t2 (i.e., the image frame acquired at time t2) from the server 3, and then receives video data including the image frame associated with time t2. The video is displayed in the video display area 52.

Furthermore, when the position of the pin P on the movement trajectory K moves from the position X2 to the position Xm (m is an integer of 3 or more) due to the drag and drop operation by the user U (see FIG. 8), the pin P is associated with the position Xm. The user terminal specifies the time tm. After that, the user terminal 4 receives the video data including the image frame associated with the time tm from the server 3, and displays the video including the video frame associated with the time tm in the video display area 52. Further, as shown in FIGS. 7 and 8, the position of the pin P, the video displayed in the video display area 52, and the position of the slider 58 on the timeline 53 are linked to each other. That is, as the position of the pin P changes, the video (video frame) displayed in the video display area 52 also changes, and the position of the slider 58 on the timeline 53 also changes. On the other hand, by moving the position of the slider 58, the video (video frame) displayed in the video display area 52 also changes, and the position of the pin P changes.

Furthermore, in this embodiment, the position of the pin P and the video (image frame) displayed in the video display area 52 are linked to each other. This is because the position of the camera 2 changes as time passes, and the video to be played changes. In this way, the position of the pin P changes automatically by continuously switching the video (image frame) displayed in the video display area 52. That is, the user terminal 4 automatically moves the pin P along the movement trajectory K and automatically moves the slider 58 on the timeline 53 as the video is played back in the video display area 52. In other words, the user terminal 4 follows the movement of the pin P on the movement trajectory K and displays a video including an image frame associated with the position indicated by the pin P in the video display area 52. Alternatively, the user terminal 4 displays a video including an image frame associated with the time indicated by the slider 58 in the video display area 52, following the movement of the slider 58 on the timeline 53.

According to this embodiment, the video display screen 50 on which the map and video are displayed simultaneously is displayed on the display unit 45 of the user terminal 4, with the movement trajectory K superimposed on the map. In this way, by viewing the movement trajectory K superimposed on the map, the user U can intuitively and from a bird's-eye view understand from which position on the map the video was captured. Furthermore, the user U can intuitively understand along what travel route on the map the video was taken. Further, the user U can easily check the image of the camera 2 on the user terminal 4 communicatively connected to the server 3 via the communication network 6 (that is, online). Therefore, it is possible to provide the information processing system 1 that allows the user to grasp the operating status of the vehicle 8 more intuitively and from a bird's-eye view online.

In addition, by looking at the pin P placed on the movement trajectory K, the user U can intuitively and bird's-eye understand from which position on the map the currently displayed image of the camera 2 was taken. can be grasped.

Further, following the movement of the pin P on the movement trajectory K, a video including a video frame associated with the position indicated by the pin P is displayed on the video display screen 50. In this way, by looking at the pin P moving along the movement trajectory K, the user U can intuitively know from which position on the map the currently displayed image of the camera 2 was taken. Moreover, it can be grasped from a bird's-eye view.

Further, the user U can select a desired position on the movement trajectory K by moving the pin P. As a result, the user U can easily and quickly check the video including the video frame associated with the selected desired position. In this way, the user U can easily check the image of the camera 2 at a desired position by performing a drag and drop operation on the pin P displayed on the map.

In addition, in the description of this embodiment, each process of steps S5 and S6 is executed by the user terminal 4 side, but these processes may be executed by the server 3 side or the camera 2 side. In this regard, after performing user authentication in step S3, the server 3 generates a movement trajectory K of the camera 2 based on the position information of the camera 2, and then exports a map covering the movement trajectory K to an external source through the API. Obtained from server 5. Thereafter, the server 3 transmits the image display data, the image of the camera 2, a map on which the movement trajectory K is superimposed, and the position information of the camera 2 to the user terminal 4. As a result, a video display screen 50 is displayed on the user terminal 4.

Further, the camera 2 may generate the movement trajectory K of the camera 2 based on the position information of the camera 2, and then transmit information indicating the movement trajectory K to the server 3. Thereafter, the server 3 may transmit the image display data, the image of the camera 2, a map on which the movement trajectory K is superimposed, and the position information of the camera 2 to the user terminal 4.

Further, in this embodiment, the map is acquired from the external server 5, but the map data may be stored in advance in the storage device 31 of the server 3.

In addition, in the description of this embodiment, the movement trajectory of the camera 2 is generated in the user terminal 4, and a map is acquired from an external server. After that, the user terminal 4 generates a video display screen in which the movement trajectory is superimposed on the map. On the other hand, all or part of these series of processes may be executed by the server 3 and/or the camera 2.

(Processing for displaying the movement trajectory during the specified period specified by the user)
Next, a process for displaying the movement trajectory K during the specified period specified by the user U will be described with reference to FIGS. 9 and 10. FIG. 9 is a flowchart illustrating a process for displaying the movement trajectory K during the specified period specified by the user U. FIG. 10 is a diagram showing an example of a video display screen 50 on which a calendar 55 on which a date can be specified is displayed.

As shown in FIG. 9, in step S10, the user terminal 4 determines the designated period through the user U's input operation. For example, as shown in FIG. 10, when the user U specifies a desired date (in this example, May 6, 2022) through the calendar 55, the user terminal 4 uses the calendar 55 to specify a desired date (May 6, 2022). 0:00 to 24:00).

Next, in step S11, the user terminal 4 transmits information regarding the specified period to the server 3 via the communication network 6. Thereafter, in response to receiving the information regarding the specified period, the server 3 retrieves the position information of the camera 2 within the specified period from among the position information of the camera 2 stored in the storage device 31, and also retrieves the position information of the camera 2 within the specified period. The video data within the specified period is extracted from the video data that has been stored. Thereafter, the server 3 transmits the position information and video data within the specified period to the user terminal 4 via the communication network 6. In this way, the user terminal 4 receives position information and video data within the specified period from the server 3 (step S12).

Next, the user terminal 4 generates a movement trajectory K of the camera 2 based on the position information within the specified period (step S13). Thereafter, the user terminal 4 displays on the display unit 45 a video display screen 50 (see FIG. 8) on which a map on which the movement trajectory K is superimposed and a video are simultaneously displayed (see FIG. 8) (step S14).

According to this embodiment, after a specified period is specified by the user U, the position information and video data of the camera 2 during the specified period are transmitted from the server 3 to the user terminal 4. Thereafter, the movement trajectory K within the specified period is displayed on the map based on the position information during the specified period. In this way, the user U can easily check the movement trajectory K and video within a desired period through input operations on the video display screen 50.

Note that in this example, when the user U specifies a specific date through the calendar 55, the specified period is determined by the user terminal 4, but the present embodiment is not limited to this. In this regard, when a time period defined by two different times is specified through an input operation by the user U, the specified period may be determined by the user terminal 4. For example, it is assumed that, as an initial state, a moving trajectory K and video in a time period from 0:00 to 24:00 on May 5, 2022 are displayed on the video display screen 50. In this case, when the time period between 10:00 and 20:00 is specified by user U's input operation, the specified period (the time period between 10:00 and 20:00 on May 5, 2022) is determined by the user terminal 4. may be done.

Furthermore, when a period defined by two different dates is specified through an input operation by the user U, the specified period may be determined by the user terminal 4. For example, when a period from May 5, 2022 to May 8, 2022 is specified through the calendar 55, the specified period (from 0:00 on May 5, 2022 to May 8, 2022) is specified by the user terminal 4. A period of 24 hours on the 8th day) may be determined.

(Processing that changes the visual aspect of the movement trajectory according to the movement speed of the camera)
Next, a process of changing the visual aspect of the movement trajectory K according to the movement speed of the camera 2 will be described below with reference to FIGS. 11 and 12. FIG. 11 is a diagram schematically showing a part of the movement trajectory K. FIG. 12 is a diagram showing a state in which the visual aspect of the line segment L1 forming the movement trajectory K has been changed. FIG. 13 is a diagram showing a state in which information related to the moving speed of the camera 2 is displayed. Note that in FIGS. 11 to 13, for convenience of explanation, illustration of the map on which the movement trajectory K is superimposed is omitted.

The movement trajectory K is composed of a plurality of line segments connecting two mutually adjacent positions. As shown in FIG. 11, a line segment L1 connecting positions X1 and X2 and a line segment L2 connecting positions X2 and X3 constitute a part of the movement trajectory K. Arrows indicating the moving direction of the camera 2 are attached to the line segments L1 and L2. For example, since the camera 2 moves from the position X1 to the position X2, an arrow indicating the moving direction in which the camera 2 moves from the position X1 to the position X2 is attached to the line segment L1. In FIG. 11, the longer the length of the predetermined line segment that constitutes the movement trajectory K, the faster the moving speed of the camera 2 associated with the predetermined line segment becomes. In other words, the length of the predetermined line segment (an example of a visual aspect) changes depending on the moving speed associated with the predetermined line segment.

For example, since the length of the line segment L1 is longer than the length of the line segment L2, the moving speed V1 of the camera 2 associated with the line segment L1 is greater than the moving speed V2 of the camera 2 associated with the line segment L2. . Here, the user terminal 4 determines the moving speeds V1 and V2 based on the position information of the camera 2. More specifically, the user terminal 4 calculates the length of the line segment L1 |X2-X1| and the time interval (t2- t1), the moving speed V1 associated with the line segment L1 is determined (V1=|x2-x1|/(t2-t1)). Similarly, the user terminal 4 calculates the length of line segment L2 |X3-X2| and the time interval (t3-t2) between time t2 when position X2 is acquired and time t3 when position Based on this, the moving speed V2 associated with the line segment L2 is determined (V2=|x3-x2|/(t3-t2)). Further, when the current position of the camera 2 is acquired by the position information acquisition unit 22 at regular time intervals (for example, 10 seconds), the moving speed of the camera 2 associated with a predetermined line segment is Directly proportional to the length of the line segment. In other words, the ratio of the moving speeds V1 and V2 and the length of the line segments L1 and L2 are equal (V1:V2=length of the line segment L1:length of the line segment L2). In this way, according to the present embodiment, the user U can easily grasp the moving speed of the vehicle 8 (camera 2) in each line segment section by looking at the length of each line segment that makes up the movement trajectory K. be able to.

Further, as shown in FIG. 12, the display color (an example of a visual aspect) of some line segments constituting the movement trajectory K may be changed depending on the movement speed of the camera 2. More specifically, the user terminal 4 changes the display color of the predetermined line segment based on the comparison result between the moving speed associated with the predetermined line segment and a predetermined speed (for example, speed limit). You may. In the example shown in FIG. 12, the user terminal 4 may change the display color of the line segment L1 when the moving speed V1 associated with the line segment L1 is higher than the speed limit (legal speed, etc.). For example, if the display color of the movement trajectory K is the first display color as an initial state, even if the display color of the line segment L1 is changed from the first display color to a second display color different from the first display color. good. As described above, according to the present embodiment, by visually recognizing the line segment L1 whose display color has been changed, the user U can recognize that the vehicle 8 was traveling in excess of the speed limit in the section of the line segment L1. It can be easily understood.

Further, as shown in FIG. 13, information related to the moving speed of the camera 2 may be displayed on a map (not shown in FIG. 13). More specifically, the information related to the moving speed of the camera may be displayed on the map in a state where the information is visually associated with the line segments forming the moving trajectory K. In the example shown in FIG. 13, when the moving speed V1 associated with the line segment L1 is larger than the speed limit, information related to the moving speed V1 (i.e., information indicating that the moving speed V1 exceeds the speed limit) ) may be displayed on the map while being visually associated with the line segment L1. Additionally, when the vehicle 8 (camera 2) has stopped for a certain period of time at position X3, information indicating that the vehicle 8 has stopped is displayed on the map in a state that is visually associated with position X3. may be done. As described above, according to the present embodiment, the user U can easily grasp the information related to the moving speed of the vehicle 8 by viewing the information related to the moving speed of the vehicle 8 displayed on the map. I can do it.

(Process of deciding to exclude some line segments from the movement trajectory K)
Next, a process for determining to exclude some line segments from the movement trajectory K will be described below with reference to FIG. In the example shown in FIG. 14, the movement trajectory K does not include a line segment connecting the mutually adjacent positions X6 and X7. In this respect, the user terminal 4 determines, based on the position information of the camera 2, that the time interval between the time t6 when the position X6 is acquired and the time t7 when the position X7 is acquired is a predetermined time interval (for example, 15 seconds), it may be determined to exclude the line segment connecting the positions X6 and X7 from the movement trajectory K. For example, if a tunnel exists in section S between position X6 and position Unable to receive radio waves from. Therefore, the camera 2 cannot acquire position information in the section S. Furthermore, in a case where the information indicating the position X6 is acquired by the GPS receiver before the vehicle 8 enters the tunnel, and the information indicating the position X7 is acquired by the GPS receiver after the vehicle 8 exits the tunnel, The time interval between time t6 when position X6 is acquired and time t7 when position X7 is acquired becomes large.

In such a case, there is a possibility that the line segment connecting the position X6 and the position X7 does not match the actual route of the tunnel in the section S (particularly, the route of the tunnel including a curve). In this way, as shown in FIG. 14, by excluding the line segment connecting position X6 and position ) can be displayed on the map.

Further, as shown in FIG. 15, a reduced screen of the video (video frame) may be displayed on the map. More specifically, when the user U specifies an arbitrary position on the movement trajectory K with the mouse pointer, even if a reduced screen of the video frame associated with the specified position is displayed in the display area 59. good. In the example shown in FIG. 15, when a position near the pin P is specified by the user U, a reduced screen of the video frame associated with the specified position is displayed in the display area 59.

(Display event information)
Next, the process of displaying event information on a map will be described below with reference to FIG. FIG. 16 is a diagram showing an example of the video display screen 50 on which accident information is displayed on a map as event information. In this regard, the server 3 identifies the accident (an example of a predetermined event) and the location where the accident occurred based on the video data and position information of the camera 2. More specifically, the server 3 identifies the occurrence of an accident from the audio information included in the video data, and also identifies the time at which the accident occurred. Next, the server 3 identifies the location where the accident occurred based on the time when the accident occurred and the position information of the camera 2. Thereafter, the server 3 transmits accident information and information indicating the location where the accident occurred to the user terminal 4. The user terminal 4 displays accident information on a map based on the accident information and information indicating the location where the accident occurred. For example, as shown in FIG. 16, if the place where the accident occurred is position X8, the accident information is displayed on the map in a state where it is visually associated with position X8 on the movement trajectory K. Good too.

(Processing that only sends location information to the user terminal during the period when the camera is moving)
Next, with reference to FIGS. 17 to 20, a process of transmitting only position information during a period in which the camera 2 is moving to the user terminal 4 will be described below. FIG. 17 is a flowchart for explaining the process of transmitting only the position information during the period Ta during which the camera 2 is moving to the user terminal 4. FIG. 18 is a diagram showing the relationship between the period Ta during which the camera 2 is moving or the period Tb during which the camera 2 is not moving and the position information of the camera 2. FIG. 19 is a schematic diagram showing an example of a movement trajectory K1 generated based on both position information in a period Ta in which the camera 2 is moving and position information in a period Tb in which the camera 2 is not moving. FIG. 20 is a schematic diagram showing an example of a movement trajectory K2 generated only based on position information during a period Ta during which the camera 2 is moving.

As shown in FIG. 17, in step S20, the server 3 receives movement data from the camera 2 via the communication network 6 indicating whether the camera 2 (vehicle 8) is moving. Here, the movement data may be acceleration data. The acceleration data is, for example, data indicating a change in the value of acceleration of the camera 2 over time. That is, the acceleration data is data indicating the value of the acceleration of the camera 2 at each time. Further, as shown in the process of step S4 in FIG. 5, the server 3 may receive the movement data from the camera 2 together with the position information and orientation information of the camera 2.

Next, the server 3 determines whether the camera 2 is moving based on the received movement data (step S21), and determines the period Ta during which the camera 2 is moving based on the movement data. A period Tb during which the character is not moving is specified (step S22). Thereafter, the server 3 classifies the position information in the period Ta and the position information in the period Tb based on the information regarding the periods Ta and Tb and the position information of the camera 2 (see FIG. 18). In step S23, the server 3 transmits the position information in the period Ta to the user terminal 4, but does not transmit the position information in the period Tb to the user terminal 4. In this way, the user terminal 4 receives only the position information in the period Ta from the server 3, and then generates the movement trajectory K2 (see FIG. 20) based only on the position information in the period Ta.

In this respect, for example, as shown in FIGS. 19 and 20, the movement trajectory K1 generated based on both the position information in the period Ta and the position information in the period Tb is generated based only on the position information in the period Ta. The trajectory is disordered compared to the moving trajectory K2. The reason for this is that the positions X13, X14, and X15 acquired during the period Tb in which the camera 2 is not moving are different from each other due to a decrease in the accuracy of GPS. In other words, the positions X13, X14, and X15 during the period Tb in which the camera 2 is not moving should normally be constant, but due to the decrease in GPS accuracy, the positions X13, X14, and X15 are actually constant. It is not. For this reason, when the movement trajectory is generated using the position information in the period Tb, there is a possibility that the movement trajectory may be disturbed as shown in FIG. 19.

According to the present embodiment, since the position information during the period Tb is not transmitted to the user terminal 4, the movement trajectory of the camera 2 is disturbed due to the accuracy of GPS in a situation where the camera 2 is not actually moving. This can be suitably prevented.

Although the embodiments of the present invention have been described above, the technical scope of the present invention should not be interpreted to be limited by the description of the embodiments. This embodiment is an example, and those skilled in the art will understand that various modifications can be made within the scope of the invention as set forth in the claims. The technical scope of the present invention should be determined based on the scope of the invention described in the claims and the scope of equivalents thereof.

According to the present embodiment, since the camera 2 is mounted on the vehicle 8 (an example of a moving object), the user U can intuitively and bird's-eye view the movement status of the vehicle 8 through the video display screen 50 on which the movement trajectory is displayed. can be understood in terms of Similarly, when the camera 2 is attached to a person (particularly a pedestrian or a runner), the user U can intuitively monitor the running situation of the pedestrian or the like through the video display screen 50 on which the movement trajectory is displayed. Moreover, it can be grasped from a bird's-eye view.

1: Information processing system, 2: Camera, 3: Server, 4: User terminal, 5: External server, 6: Communication network, 8: Vehicle, 20: Control unit, 21: Storage device, 22: Location information acquisition unit, 24: inertial sensor, 25: communication unit, 26: input operation unit, 27: imaging unit, 28: audio input unit, 29: PTZ mechanism, 30: control unit, 31: storage device, 33: communication unit, 34: input Operation unit, 35: Display unit, 40: Control unit, 41: Storage device, 43: Communication unit, 44: Input operation unit, 45: Display unit, 50: Video display screen, 51: Map display area, 52: Video display Area, 53: Timeline, 55: Calendar, 58: Slider, 59: Display area, 63: Display area

Claims (19)

An information processing system that displays images captured by a camera on a user terminal via a server,
Obtaining an image shot by the camera,
obtaining position information indicating a plurality of positions of the camera within a predetermined period;
generating a movement trajectory composed of line segments connecting a plurality of positions of the camera based on the position information;
get the map,
displaying the map and the video on the user terminal with the movement trajectory superimposed on the map;
Information processing system. The information processing system includes:
Selecting a predetermined position on the movement trajectory in response to a user's input operation on the user terminal,
displaying a video including an image frame associated with the selected predetermined position;
The information processing system according to claim 1. The information processing system includes:
displaying pins placed on the movement trajectory;
displaying a video including an image frame associated with the location indicated by the pin;
The information processing system according to claim 1. The information processing system includes:
automatically moving the pin along the movement trajectory;
Displaying an image frame associated with a position indicated by the pin in accordance with the movement of the pin on the movement trajectory;
The information processing system according to claim 3. The information processing system includes:
determining the predetermined period according to a user's input operation on the user terminal;
The information processing system according to any one of claims 1 to 4. The information processing system includes:
determining the predetermined period of time through designation of a time zone defined by two different times, a specific date, or a period defined by two different dates;
The information processing system according to claim 5. The information processing system includes:
obtaining a moving speed of the camera associated with a predetermined line segment of the moving trajectory;
determining a visual aspect of the predetermined line segment according to the movement speed;
The information processing system according to any one of claims 1 to 4. The location information is
first position information indicating a first position;
second position information indicating a second position different from the first position and obtained after the first position information;
including;
The movement trajectory includes a line segment connecting the first position and the second position,
The information processing system includes:
A moving speed of the camera associated with the line segment is determined based on the length of the line segment and the time interval between the acquisition time of the first position information and the acquisition time of the second position information. ,
The information processing system according to any one of claims 1 to 4. The location information is
first position information indicating a first position;
second position information indicating a second position different from the first position and obtained after the first position information;
including;
When the time interval between the acquisition time of the first position information and the acquisition time of the second position information is larger than a predetermined time interval, the line segment connecting the first position and the second position is Not included in the movement trajectory,
The information processing system according to any one of claims 1 to 4. The information processing system includes:
determining a first position on the movement trajectory in response to a user's input operation on the user terminal;
displaying a first image frame associated with the first location on the map;
The information processing system according to any one of claims 1 to 4. The information processing system includes:
Based on the video and the location information, specifying the occurrence of a predetermined event and specifying the location where the predetermined event occurred;
displaying information related to the predetermined event on the map based on the location where the predetermined event occurs;
The information processing system according to any one of claims 1 to 4. The information processing system includes:
obtaining a moving speed of the camera associated with a predetermined line segment on the moving trajectory;
changing the visual aspect of the predetermined line segment based on a comparison result between the moving speed and a predetermined speed;
The information processing system according to any one of claims 1 to 4. The information processing system includes:
changing the visual aspect of the predetermined line segment when the moving speed is greater than a speed limit;
The information processing system according to claim 12. The information processing system includes:
obtaining a moving speed of the camera associated with a first position on the moving trajectory;
Displaying information related to the moving speed on the map based on a comparison result between the moving speed and a predetermined speed;
The information processing system according to any one of claims 1 to 4. The information processing system includes:
receiving data from the camera indicating whether the camera is moving;
determining whether the camera is moving based on the data;
If it is determined that the camera is moving within a first period, transmitting the position information within the first period to the user terminal;
not transmitting the position information within the second period to the user terminal if it is determined that the camera has not moved within the second period;
The information processing system according to any one of claims 1 to 4. The camera is mounted on a moving object or attached to a person,
The information processing system according to any one of claims 1 to 4. An information processing device communicably connected to a camera and a user terminal via a communication network,
The information processing device includes:
a processor;
a memory for storing computer readable instructions;
When the computer readable instructions are executed by the processor, the information processing device:
receiving an image captured by the camera;
receiving position information indicating a plurality of positions of the camera within a predetermined period;
transmitting the location information and the video to the user terminal;
The map and the video are displayed on the user terminal with the movement trajectory of the camera superimposed on the map,
The movement trajectory is generated based on the position information and is composed of line segments connecting a plurality of positions of the camera.
Information processing device. An information processing method executed by a user terminal, the method comprising:
The user terminal is communicatively connected to a server via a communication network,
the server is communicatively connected to the camera via the communication network;
The information processing method includes:
receiving an image taken by the camera from the server;
receiving position information from the server indicating a plurality of positions of the camera within a predetermined period;
generating a movement trajectory composed of line segments connecting a plurality of positions of the camera based on the position information;
a step of obtaining a map;
displaying the map and the video on the user terminal with the movement trajectory superimposed on the map;
including,
Information processing method. A program for causing the user terminal to execute the information processing method according to claim 18.

Images