JP2017208631A - Image display system, information processing apparatus, imaging apparatus, program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image display system capable of evaluating the movement of a moving object easily.SOLUTION: An image display system having an information processing device, a wide angle imaging device capable of imaging the horizontal perimeter, and one or more measuring apparatuses for measuring the movement of one or more moving objects, respectively, further includes acquisition means for acquiring image data generated when the wide angle imaging device images the one or more objects, and one or more pieces of measurement data generated when the one or more measuring apparatuses measure movement of the one or more moving objects, respectively, first mapping means for specifying the positions of the one or more moving objects in the image data, respectively, and mapping the positions to the image data, second mapping means for mapping the one or more pieces of measurement data, respectively, to the one or more pieces of image data, and display means for displaying the image data, and one or more pieces of measurement data mapped to the image data, based on the positions mapped to the image data.SELECTED DRAWING: Figure 1

Description

  The present invention relates to an image display system, an information processing apparatus, an imaging apparatus, and a program.

  2. Description of the Related Art A motion capture system that analyzes a person's walking motion is conventionally known. In the motion capture system, for example, a person wearing a predetermined marker at a joint position or the like of a body is imaged with a camera, and the movement of the person can be analyzed by tracking the movement of the marker with a tracker.

  Here, the motion capture system requires a large number of markers in order to capture human movements from various angles, and there is a tendency that it takes time and cost to prepare for imaging of walking motion and analysis after imaging. In addition, it is difficult to analyze walking motions except in a special environment such as a laboratory.

  For these problems, a technique is known that can analyze walking motion with a relatively simple configuration using captured images of a pedestrian with a plurality of marks attached to a predetermined part of the body (for example, Patent Documents). 1).

  However, the above-described conventional technique has a problem that the walking motion that can be imaged is limited. This is because the position and orientation of the camera are fixed while the person gradually moves by walking. For example, when the camera is fixed, it is possible to capture a walking motion other than the walking motion of a pedestrian walking in a straight line (for example, a walking motion of a pedestrian walking while turning around the camera). Have difficulty.

  Embodiments of the present invention have been made in view of the above points, and an object of the present invention is to provide an image display system that can easily evaluate the movement of a moving object.

  In order to achieve the above object, an embodiment of the present invention includes an information processing device, a wide-angle imaging device capable of imaging the surroundings in the horizontal direction, and one or more measurement devices that respectively measure the motion of one or more moving objects. The image display system includes: image data generated by the wide-angle imaging device capturing the one or more objects; and the one or more measuring devices respectively measuring the operations of the one or more objects. Acquisition means for acquiring one or more measurement data generated by the first and second association means for identifying the positions of the one or more objects in the image data and associating the identified positions with the image data; Based on the second association means for associating each of the one or more measurement data with the image data, and the position associated with the image data, It includes an image data, and a display means for displaying on the display device and the one or more measurement data associated with the image data.

  According to the embodiment of the present invention, it is possible to provide an image display system that can easily evaluate the movement of a moving object.

It is a figure explaining an example of schematic operation | movement of an image display system. It is a figure which shows an example of the system configuration | structure of an image display system. It is a figure which shows an example of the hardware constitutions of a server and a display apparatus. It is a figure which shows an example of the hardware constitutions of an acceleration measuring device. It is a figure which shows an example of the hardware constitutions of an omnidirectional image imaging device. It is a figure which shows an example of the functional block of a server, a display apparatus, an acceleration measuring apparatus, and an omnidirectional image imaging device. It is a figure explaining an example of a pedestrian's walk. It is a sequence diagram explaining an example of the whole operation | movement of an image display system. It is a sequence diagram explaining the modification (the 1) of the whole operation | movement of an image display system. It is a sequence diagram explaining the modification (the 2) of the whole operation | movement of an image display system. It is a sequence diagram explaining the modification (the 3) of the whole operation | movement of an image display system. FIG. 10 is a sequence diagram illustrating a modification (No. 4) of the overall operation of the image display system. It is a sequence diagram explaining the modification (the 5) of the whole operation | movement of an image display system. It is a flowchart explaining an example of a display image creation process. It is a figure which illustrates typically an example of specification of a rectangular field in which a pedestrian is reflected. It is a figure explaining an example of the position of the pedestrian matched for every display frame data. It is a flowchart explaining the other example (the 1) of a display image creation process. It is a figure explaining the other example (the 1) of the position of the pedestrian matched for every display frame data. 12 is a flowchart for explaining another example (part 2) of the display image creation process. It is a figure explaining the other example (the 2) of the position of the pedestrian matched for every display frame data. It is a figure which shows an example of the display image data and measurement data which were displayed on the display apparatus. It is a flowchart which shows an example of the process which displays the range which the pedestrian is reflected in an image data column. It is a figure which shows the other example of the system configuration | structure of an image display system. It is a sequence diagram explaining an example of the whole operation | movement of an image display system in case an acceleration measuring apparatus displays measurement data and display image data.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

<Outline of Image Display System 100>
First, a schematic operation of the image display system 100 according to the present embodiment will be described. FIG. 1 is a diagram illustrating an example of a schematic operation of the image display system 100 according to the present embodiment.

  In the image display system 100 according to the present embodiment, the omnidirectional image capturing device 60 is installed, and the omnidirectional image capturing device 60 captures the pedestrian 200A and the pedestrian 200B walking around the line L. To do.

  Further, the pedestrian 200A is equipped with an acceleration measuring device 50A on the body, and the acceleration measuring device 50A measures the acceleration generated during walking of the pedestrian 200A. Similarly, an acceleration measuring device 50B is attached to the body of the pedestrian 200B, and the acceleration measuring device 50B measures the acceleration that occurs during walking of the pedestrian 200B.

  (1) The omnidirectional image capturing device 60 transmits image data generated by capturing the pedestrian 200 </ b> A and the pedestrian 200 </ b> B to the server 10.

  Further, the acceleration measuring device 50A transmits the measurement data A generated by measuring the acceleration generated during walking of the pedestrian 200A to the server 10. Similarly, the acceleration measuring device 50B transmits the measurement data B generated by measuring the acceleration generated during walking of the pedestrian 200B to the server 10.

  The imaging start time of the image data is synchronized with the measurement start times of the measurement data A and the measurement data B. In addition, the image data capture end time and the measurement data A and measurement data B measurement end times are synchronized.

  (2) The server 10 creates display image data for displaying the walking motions of the pedestrian 200A and the pedestrian 200B shown in the image data. Then, the server 10 synchronizes the time axes of the display image data and the measurement data A and synchronizes the time axes of the display image data and the measurement data B.

  (3) The server 10 transmits display image data, measurement data A, and measurement data B to the display device 30. Thereby, the display device 30 can display the display image data, the measurement data A, and the measurement data B on the same time axis.

  The display image data is, for example, image data for switching and displaying the pedestrian 200A and the pedestrian 200B shown in the image data on the display device 30.

  According to the configuration shown in FIG. 1, the omnidirectional imaging device 60 is used, and the pedestrian 200A and the pedestrian 200B walking around the pedestrian 200A (hereinafter, when the pedestrian 200A and the pedestrian 200B are not distinguished from each other). Is represented as “pedestrian 200”).

  For this reason, in the image display system 100 according to the present embodiment, not only the pedestrian 200 walking linearly but also the pedestrian 200 walking while turning right or turning left can be imaged. It is known that the pedestrian 200 easily loses its center of gravity when turning. Therefore, in the image display system 100 according to the present embodiment, in addition to the case of moving linearly, it becomes possible to effectively analyze the walking motions of a plurality of pedestrians 200 that turn.

  In addition, although FIG. 1 demonstrated the case where there were two pedestrians 200, the number of pedestrians 200 may be three or more. Moreover, the pedestrian 200 may be one person. Further, for example, among the plurality of pedestrians 200, there may be a pedestrian 200 not wearing the acceleration measuring device 50.

  Since the omnidirectional image capturing device 60 can capture the side surface of the pedestrian 200 without moving the camera, the omnidirectional image capturing device 60 can capture how the body of the pedestrian 200 shakes in the front-rear direction. For this reason, according to the image display system 100 according to the present embodiment, an instructor or the like of walking can check and evaluate the walking motion of the pedestrian 200 with an image, and can guide improvement points. .

  In the image display system 100 according to the present embodiment, the measurement data A and the measurement data B in which the acceleration of the pedestrian 200 is measured (hereinafter, when the measurement data A and the measurement data B are not distinguished from each other, “measurement data Is displayed on the same time axis as the display image data. For this reason, when a leader etc. discover the walking motion which should be improved, it can be proved by measurement data that the walking motion is not good.

  In addition, in the image display system 100 according to the present embodiment, a leader or the like finds measurement data that is not good by comparing the measurement data that is prepared in advance with good measurement data and the measurement data of the pedestrian 200. You can also Since the measurement data of the pedestrian 200 that is not good is associated with the display image data, the instructor or the like can specify the walking motion to be improved from the display image data.

  Therefore, by using the image display system 100 according to the present embodiment, a leader or the like can evaluate the walking motion using the measurement data and the display image data in a complementary manner.

  Hereinafter, when the acceleration measuring device 50A and the acceleration measuring device 50B are not distinguished, they are expressed as “acceleration measuring device 50”. In the following description, it is assumed that the image data and the display image data are moving image data. Therefore, the image data includes a plurality of frame data that is data of an image (frame image) for one frame. Similarly, the display image data includes a plurality of display frame data, which is frame image data. However, the image data is not limited to moving images, and may be image data of still images (still image data).

  Here, the pedestrian 200 is an object that moves forward, backward, laterally, and the like by an action such as walking, but is not limited to a person, and may be, for example, an animal, a robot, or the like. Also, walking may include running or may walk backwards. Further, the object to be moved may be a wheelchair, a person wearing a prosthetic leg, a bicycle, a stilt, a tricycle, or the like. Hereinafter, for the sake of convenience, it is assumed that the moving object is “pedestrian 200”.

  The measurement data is information used to evaluate the movement of the moving object, but is not limited to acceleration, and may be, for example, angular velocity, position, or direction.

  Further, the display image data and the measurement data are associated with each other when one or more display frame data among the display frame data included in the display image data is determined, It means that two or more measured values are determined. Alternatively, when display frame data and a measurement value at substantially the same time are specified, if the time, display frame data, or one of the measurement values is determined, the rest is determined.

  The instructor refers to a person who can confirm the walking motion of the pedestrian 200 with an image, evaluate the walking motion, and plan improvement points. Specific examples include holders of qualifications such as public health nurses, clinical laboratory technicians, and prosthetic limb orthoses.

  Further, even if the person is not a person who can evaluate the walking motion and plan improvement points, the image display system 100 can be operated, and the instructor may be called an operator, a user, a user, or the like. .

<System configuration>
Next, the system configuration of the image display system 100 according to the present embodiment will be described. FIG. 2 is a diagram illustrating an example of a system configuration of the image display system 100 according to the present embodiment.

  An image display system 100 according to the present embodiment includes a server 10, a display device 30, an acceleration measuring device 50, and an omnidirectional image capturing device 60, which are connected to each other via a network N so as to communicate with each other.

  The network N is a LAN (Local Area Network) constructed in a facility where the image display system 100 is arranged. The network N may include the Internet, a telephone line network, and the like. Further, the network N may be constructed by either wired or wireless, and may be a combination of wired and wireless.

  The omnidirectional image capturing device 60 is an image capturing device (camera) that captures a 360 ° circumference called an omnidirectional image or an omnidirectional image (hereinafter referred to as an “omnidirectional image”). The omnidirectional image capturing device 60 includes an optical system to be described later, so that an omnidirectional image can be obtained by a single imaging operation. Since the omnidirectional image includes an image of 360 ° around the blind spot, an instructor or the like can selectively display an arbitrary range on the display or the like for viewing.

  In the present embodiment, the omnidirectional image capturing device 60 may not have an image capturing range of 360 ° in the latitude direction (the vertical direction of the image). This is because the pedestrian 200 is not captured even when the ceiling or floor is imaged.

  Further, the omnidirectional image capturing device 60 may not have an imaging range of 360 ° in the longitude direction (the horizontal direction of the image). This is because an image of the entire celestial sphere image capturing device 60 may not be necessary for the instructor to analyze the walking motion of the pedestrian 200.

  As the omnidirectional image capturing device 60, an image capturing device that obtains an omnidirectional image by connecting a plurality of images obtained by capturing a plurality of rectangular ranges at a certain angle of view may be used.

  The acceleration measuring device 50 is a device that measures the acceleration of the pedestrian 200 and obtains measurement data including a measured value indicating the measured acceleration. The acceleration is preferably measured on each of the three axes of vertical direction acceleration, traveling direction acceleration, and left-right direction acceleration, but may be acceleration in at least one direction.

  A dedicated sensor (acceleration sensor) for measuring acceleration also exists as the acceleration measuring device 50, but the acceleration measuring device 50 may be a device having an acceleration sensor. Specifically, the acceleration measuring device 50 is, for example, a smartphone, a mobile phone, a tablet terminal, a notebook PC (Personal Computer), a wearable terminal (a PC mounted as a head mounted display, sunglasses, a wristwatch, or the like), a pedometer, or the like. There may be.

  Further, the acceleration measuring device 50 may have a function of measuring, for example, angular velocity, position, azimuth, and the like in addition to acceleration.

  The angular velocity can be measured using, for example, a gyro sensor. The position can be measured using, for example, a GPS (Global Positioning System) satellite, IMES (Indoor MEssaging System), or a beacon. The azimuth can be measured, for example, using an azimuth sensor that detects the east, west, south, and north directions, for example, based on the north. Any of these can be used for evaluating the walking motion of the pedestrian 200 as well as the acceleration.

  In the present embodiment, the acceleration measurement device 50 is described as measuring acceleration. However, the present invention is not limited to this, and the acceleration measurement device 50 uses various measurement values used for evaluation of walking motion, such as angular velocity, position, and orientation. The included measurement data may be transmitted to the server 10.

  The server 10 is an information processing device that receives image data from the omnidirectional image capturing device 60 and creates display image data. Further, the server 10 synchronizes the time axis between the created display image data and the measurement data received from the acceleration measuring device 50. The function of the server 10 may be included in the acceleration measuring device 50 or the display device 30.

  The display device 30 is an information processing device used as a terminal device that allows a leader or the like to display display image data and measurement data. Specifically, for example, a PC, a smartphone, a tablet terminal, a mobile phone, a car navigation terminal, a wearable terminal, a camera, an electronic blackboard, a projector, a game machine, an MFP (Multifunction Peripheral / Printer / Product), and the like.

<Hardware configuration>
Next, the hardware configuration of the server 10, the display device 30, the acceleration measuring device 50, and the omnidirectional image capturing device 60 according to the present embodiment will be described.

  FIG. 3 is a diagram illustrating an example of a hardware configuration of the server 10 and the display device 30. Although the hardware configurations of the server 10 and the display device 30 illustrated in FIG. 3 are illustrated as examples arranged in one housing, the present invention is not limited thereto. The hardware configurations of the server 10 and the display device 30 may be distributed in, for example, a plurality of cases so that the respective hardware can communicate with each other. Since the server 10 and the display device 30 have the same hardware configuration, the hardware configuration of the server 10 will be mainly described below.

  The server 10 includes a central processing unit (CPU) 101, a read only memory (ROM) 102, a random access memory (RAM) 103, a hard disk drive (HDD) 105, and a media drive 107. The server 10 also includes a display 108, a network I / F 109, a keyboard 111, a mouse 112, and an optical drive 114. Each of these pieces of hardware is connected to each other by a bus 110.

  The CPU 101 executes an image processing program stored in an HD (Hard Disk) 104 and controls the overall operation of the server 10. The ROM 102 stores a program used to drive the CPU 101 such as an IPL (Initial Program Reader). The RAM 103 is a main storage device used as a work area for the CPU 101. The HD 104 is a storage device equipped with a nonvolatile memory.

  The HDD 105 controls reading or writing of various data with respect to the HD 104 according to the control of the CPU 101. The display 108 displays various information such as a cursor, menu, window, character, or image. The network I / F 109 is an interface with the network N. The display 108 of the server 10 may be the display device 30.

  The keyboard 111 and the mouse 112 are input / output devices. The keyboard 111 includes a plurality of keys for inputting characters, numerical values, various instructions, and the like, and receives input from the plurality of keys. The mouse 112 receives movement of the mouse pointer, selection and execution of various instructions, selection of a processing target, and the like.

  The media drive 107 controls reading or writing of various data with respect to the recording medium 106 such as a flash memory. The optical drive 114 controls reading or writing of various data with respect to a CD (Compact Disc) 113 as an example of a removable recording medium.

  Note that the above-described image processing program may be recorded and distributed on a computer-readable recording medium such as the recording medium 106 or the CD 113, for example, in a file in an installable format or an executable format. For example, the image processing program may be distributed in a form downloaded from a predetermined server device or the like.

  The server 10 and the display device 30 according to the present embodiment can implement various processes described later by having the hardware configuration illustrated in FIG. 3.

  FIG. 4 is a diagram illustrating an example of a hardware configuration of the acceleration measuring device 50. In addition, in FIG. 4, although the hardware configuration in case the acceleration measuring apparatus 50 is a smart phone or a tablet terminal is shown as an example, the hardware configuration of the acceleration measuring apparatus 50 is not restricted to this.

  The acceleration measuring apparatus 50 includes a CPU 201, ROM 202, RAM 203, flash memory 204, acceleration / direction sensor 205, gyro sensor 206, and media drive 207. The acceleration measuring device 50 includes a voice input device 210, a voice output device 211, a communication device 212, a GPS receiver 213, a display 214, and a touch panel 215. These pieces of hardware are connected to each other via a bus 209.

  The CPU 201 controls the overall operation of the acceleration measuring device 50 by executing a program stored in the flash memory 204. The ROM 202 stores IPL and basic input / output programs. A RAM 203 is a main storage device used as a work area when the CPU 201 executes a program.

  The flash memory 204 is a nonvolatile storage device that stores programs executed by the acceleration measuring device 50 and various data. The program is, for example, an operating system, a device program executed by the acceleration measuring device 50, or the like.

  The acceleration / orientation sensor 205 has a function as an electronic magnetic compass for detecting geomagnetism, a function for measuring acceleration in three axial directions, and the like. The gyro sensor 206 detects an angular velocity when the acceleration measuring device 50 rotates with respect to the x axis, the y axis, or the z axis. The rotation angle of each axis is called a yaw angle, a pitch angle, and a roll angle.

  The media drive 207 controls reading or writing (storage) of data with respect to a recording medium 208 such as a flash memory.

  The voice input device 210 is a microphone that converts voice into a voice signal. The audio output device 211 is a speaker that converts an audio signal into audio. The communication device 212 performs communication using a wireless communication signal using the antenna 212a. The communication device 212 may be a LAN card connected to the LAN.

  The GPS receiver 213 detects position information (for example, latitude, longitude, altitude, etc.) of the acceleration measuring device 50 by a GPS satellite or IMES as indoor GPS.

  The display 214 displays various information such as a cursor, menu, window, character, or image. The touch panel 215 is superimposed on the display 214 and detects a touch position (coordinates) on the display 214 in response to an operation with a finger, a touch pen, or the like.

  Note that the above-described apparatus program may be recorded in a computer-readable recording medium such as the recording medium 208 and distributed, for example, in an installable or executable file. The device program may be distributed, for example, in a form downloaded from a predetermined server device or the like.

  The acceleration measuring apparatus 50 according to the present embodiment can realize various processes described later by having the hardware configuration shown in FIG.

  FIG. 5 is a diagram illustrating an example of a hardware configuration of the omnidirectional image capturing apparatus 60. In FIG. 5, as an example, the omnidirectional image pickup device 60 is described as an image pickup device using two image pickup devices, but the number of image pickup devices may be three or more. Further, the omnidirectional image capturing device 60 is not necessarily a dedicated device for capturing an omnidirectional image. By attaching a retrospective omnidirectional image capturing unit to an ordinary digital camera, a smartphone, or the like. The omnidirectional image pickup device 60 may have substantially the same function.

  The omnidirectional image capturing device 60 includes an image capturing unit 301, an image processing unit 304, an image capturing control unit 305, a microphone 308, a sound processing unit 309, an audio output device 319, and a CPU 311. The omnidirectional image capturing device 60 includes a ROM 312, an SRAM (Static RAM) 313, a DRAM (Dynamic RAM) 314, an operation device 315, an external I / F 316, a communication device 317, and an electronic compass 318.

  The imaging unit 301 includes wide-angle lenses (so-called fish-eye lenses) 302a and 302b each having an angle of view of 180 ° or more for forming a hemispherical image, and two imaging elements 303a provided corresponding to the wide-angle lenses. And 303b.

  The image sensors 303a and 303b each convert an optical image obtained by the corresponding wide-angle lens 302a or 302b into image data of an electrical signal and output it, and a timing for generating a horizontal or vertical synchronization signal, a pixel clock, and the like of the image sensor. A generation circuit and the like. The image sensors 303a and 303b include a register group in which various commands, parameters, and the like necessary for the operation of these image sensors are set. Examples of the image sensor include a CMOS (Complementary Metal Oxide Semiconductor) sensor and a CCD (Charge Coupled Device) sensor.

  The imaging elements 303a and 303b of the imaging unit 301 are connected to the image processing unit 304 by a parallel I / F bus. In addition, the imaging elements 303a and 303b of the imaging unit 301 are connected to the imaging control unit 305 by a serial I / F bus (I2C bus or the like).

  The image processing unit 304 and the imaging control unit 305 are connected to the CPU 311 via the bus 310. In addition, a ROM 312, an SRAM 313, a DRAM 314, an operation device 315, an external I / F 316, a communication device 317, an electronic compass 318, and the like are connected to the bus 310.

  The image processing unit 304 takes in the image data output from the image sensors 303a and 303b through the parallel I / F bus, performs predetermined processing on each image data, and then combines these image data. Create image data of equirectangular images.

  In general, the imaging control unit 305 sets a command or the like in a register group of the imaging elements 303a and 303b using the I2C bus with the imaging control unit 305 as a master device and the imaging elements 303a and 303b as slave devices. At this time, necessary commands and the like are received from the CPU 311. In addition, the imaging control unit 305 also uses the I2C bus to capture the status data of the register groups of the imaging elements 303a and 303b and send it to the CPU 311.

  The imaging control unit 305 instructs the imaging elements 303a and 303b to output image data at the timing when the shutter button of the operation device 315 is pressed. Some omnidirectional image capturing devices 60 may have a preview display function by a display or a function corresponding to a moving image display. In this case, output of image data from the image sensors 303a and 303b is continuously performed at a predetermined frame rate.

  In addition, the imaging control unit 305 performs control to synchronize the output timing of the image data of the imaging elements 303a and 303b in cooperation with the CPU 311. In the present embodiment, the omnidirectional imaging device 60 is not provided with a display device, but may be provided with a display device.

  The microphone 308 converts sound into sound data. The sound processing unit 309 takes in the sound data output from the microphone 308 through the I / F bus, performs predetermined processing on the sound data, and creates an audio signal. The audio output device 319 is a speaker that converts an audio signal into audio.

  The CPU 311 controls the overall operation of the omnidirectional image capturing apparatus 60 and executes necessary processing. The ROM 312 stores various programs for the CPU 311. The SRAM 313 and the DRAM 314 are work memories, and store programs executed by the CPU 311 and data being processed. In particular, the DRAM 314 stores image data being processed by the image processing unit 304 and image data of an image created by a processed equirectangular projection.

  The operation device 315 is various operation buttons, a power switch, a shutter button, a touch panel having both display and operation functions, and the like. The user can input various imaging modes, imaging conditions, and the like by operating the operation buttons.

  The external I / F 316 is an interface (for example, USB I / F) with an external medium such as an SD card or a PC. Further, the external I / F 316 may be a network interface regardless of wireless or wired. The image data of the image created by the equirectangular projection stored in the DRAM 314 is recorded on an external medium via the external I / F 316, or an external device (via the external I / F 316 as necessary). For example, it may be transmitted to a PC or the like.

  The communication device 317 communicates with an external device by a wireless technology such as Wi-Fi (Wireless Fidelity), NFC, or LTE (Long Term Evolution) via an antenna 317 a provided in the omnidirectional image capturing device 60. . Image data of an image created by equirectangular projection may be transmitted to an external device via the communication device 317.

  The electronic compass 318 calculates the azimuth and tilt (Roll rotation angle) of the omnidirectional imaging device 60 from the earth's magnetism, and outputs azimuth / tilt information. This azimuth / tilt information is related information (metadata) along Exif, and is used for image processing such as image correction of a captured image, for example. The related information includes the date and time of image capturing, the data capacity of image data, and the like.

  The omnidirectional image capturing apparatus 60 according to the present embodiment has the hardware configuration shown in FIG. 5, and can implement various processes described later.

<Functional configuration>
Next, functional configurations of the server 10, the display device 30, the acceleration measuring device 50, and the omnidirectional image capturing device 60 will be described. FIG. 6 is a diagram illustrating an example of functional blocks of the server 10, the display device 30, the acceleration measuring device 50, and the omnidirectional image capturing device 60.

<< Functional configuration of server 10 >>
The server 10 includes a communication unit 11, an operation input reception unit 12, a synchronization instruction unit 13, an acceleration data processing unit 14, a display image creation unit 15, an image adjustment unit 16, and a storage / readout unit 19. Each of these units is realized by any one of the constituent elements shown in FIG. 3 operating according to an instruction from the CPU 101 according to the image processing program 1010 expanded on the RAM 103.

  Further, the server 10 includes a storage unit 1000 constructed by any one or more of the HD 104, the ROM 102, and the RAM 103 shown in FIG. The storage unit 1000 stores an image processing program 1010, a measurement data DB 1001, and an image data DB 1002. The measurement data DB 1001 stores measurement data transmitted from the acceleration measurement device 50. The image data DB 1002 stores image data transmitted from the omnidirectional image capturing device 60.

  The communication unit 11 is realized by a command from the CPU 101, a network I / F 109, and the like, and transmits and receives various data to and from the acceleration measuring device 50, the omnidirectional image capturing device 60, and the display device 30 via the network N. .

  Hereinafter, even when the server 10 communicates with the function of the communication unit 11, the description “communication via the communication unit 11” may be omitted. The same applies to the display device 30, the omnidirectional image capturing device 60, and the acceleration measuring device 50.

  The operation input receiving unit 12 is realized by a command from the CPU 101, a keyboard 111, a mouse 112, and the like, and receives various operations and inputs to the server 10 by a leader or the like.

  The synchronization instructing unit 13 is realized by an instruction from the CPU 101, and performs processing related to synchronization between measurement data and display image data. Specifically, the synchronization instruction unit 13 requests the acceleration measurement device 50 and the omnidirectional image imaging device 60 to start measurement and imaging at the same time. Similarly, the synchronization instruction unit 13 requests the acceleration measuring device 50 and the omnidirectional image capturing device 60 to end measurement and image capturing at the same time. Further, the synchronization instruction unit 13 provides absolute time for synchronization to the omnidirectional image capturing device 60 and the acceleration measuring device 50.

  The acceleration data processing unit 14 is realized by a command or the like from the CPU 101, and converts discrete measurement data into data represented by a continuous graph with respect to the time axis, for example, by smoothly combining the data.

  The display image creating unit 15 is realized by an instruction from the CPU 101 and the like, and creates display image data based on the image data received from the omnidirectional image capturing device 60.

  That is, for example, for each frame data included in the image data, the display image creation unit 15 determines the frame data, the pedestrian ID that identifies the pedestrian 200 reflected in the frame data, and the position of the pedestrian 200. Display frame data is created by associating with each other. Then, the display image creation unit 15 creates display image data so that the display frame data created in this way is included.

  The image adjustment unit 16 is realized by an instruction from the CPU 101 and the like, and synchronizes the time axes of the display image data created by the display image creation unit 15 and the measurement data acquired from the acceleration measuring device 50.

  The storage / reading unit 19 is realized by an instruction from the CPU 101, the HDD 105, and the like, and performs processing for storing various data in the storage unit 1000 and reading various data stored in the storage unit 1000.

  Hereinafter, even when the server 10 accesses the storage unit 1000, the description “read / write via the storage / reading unit 19” may be omitted. The same applies to the display device 30, the omnidirectional image capturing device 60, and the acceleration measuring device 50.

<< Functional Configuration of Display Device 30 >>
The display device 30 includes a communication unit 31, a display control unit 32, an operation input reception unit 33, an image rotation unit 34, and a storage / readout unit 39. Each of these units is realized by any one of the components shown in FIG. 3 operating according to a command from the CPU 101 according to the terminal program 3010 expanded on the RAM 103. These units are not limited to the terminal program 3010, and may be realized by, for example, a Web page (HTML (HyperText Markup Language), XML (Extensible Markup Language), JavaScript (registered trademark), etc.) transmitted by the server 10. .

  In addition, the display device 30 includes a storage unit 3000 configured by any one or more of the HD 104, the ROM 102, and the RAM 103 illustrated in FIG. The storage unit 3000 stores a terminal program 3010. The storage unit 3000 stores measurement data and display image data transmitted from the server 10.

  The communication unit 31 is realized by a command from the CPU 101, a network I / F 109, and the like, and transmits and receives various data to and from the server 10 via the network N.

  The display control unit 32 is realized by an instruction from the CPU 101 and the like, and displays the display image data and measurement data received from the server 10 on the display 108.

  The operation input receiving unit 33 is realized by a command from the CPU 101, a keyboard 111, a mouse 112, and the like, and receives various operations and inputs to the display device 30 by a leader or the like.

  The image rotation unit 34 is realized by a command from the CPU 101 or the like, and rotates display image data that is an omnidirectional image in the horizontal direction so that the range in which the pedestrian 200 is reflected is displayed on the display 108 (or Trimming).

  The storage / reading unit 39 is realized by an instruction from the CPU 101, the HDD 105, and the like, and performs a process of storing various data in the storage unit 3000 and reading out various data stored in the storage unit 3000.

≪Functional configuration of acceleration measuring device 50≫
The acceleration measuring device 50 includes a communication unit 51, an acceleration detection unit 52, a sound output unit 53, and a storage / readout unit 59. Each of these units is realized by any one of the constituent elements shown in FIG. 4 operating according to a command from the CPU 201 according to the device program 5010 expanded on the RAM 203.

  Further, the acceleration measuring apparatus 50 includes a storage unit 5000 configured by any one or more of the flash memory 204, the ROM 202, and the RAM 203 illustrated in FIG. The storage unit 5000 stores an apparatus program 5010. Further, the storage unit 5000 stores measurement data including measurement values such as acceleration measured by the acceleration measuring device 50.

  The communication unit 51 is realized by a command from the CPU 201, the communication device 212, and the like, and transmits and receives various data to and from the server 10 via the network N.

  The acceleration detection unit 52 is realized by a command from the CPU 201 and the acceleration / direction sensor 205 and the like, and measures the acceleration generated in the acceleration measuring device 50. The acceleration is measured at regular intervals or irregularly, and an acceleration that changes with time is obtained as a measurement value. Then, the acceleration detection unit 52 stores measurement data including measurement values obtained in time series in the storage unit 5000.

  The sound output unit 53 is realized by a command from the CPU 201, the sound output device 211, and the like, and outputs a sound such as a buzzer sound.

  The storage / reading unit 59 is realized by an instruction from the CPU 201, the flash memory 204, the ROM 202, the RAM 203, and the like, and stores various data in the storage unit 5000 and reads out various data stored in the storage unit 5000. I do.

≪Functional configuration of omnidirectional imaging device 60≫
The omnidirectional image capturing apparatus 60 includes a communication unit 61, an image capturing unit 62, a sound output unit 63, and a storage / readout unit 69. Each of these units is realized by any one of the constituent elements illustrated in FIG. 5 operating according to a command from the CPU 311 according to the imaging program 6010 expanded in the SRAM 313 or the DRAM 314.

  Further, the omnidirectional image capturing device 60 includes a storage unit 6000 constructed by one or more of the ROM 312, the SRAM 313, and the DRAM 314 shown in FIG. The storage unit 6000 stores an imaging program 6010. The storage unit 6000 stores image data captured by the omnidirectional image capturing device 60.

  The communication unit 61 is realized by an instruction from the CPU 311, a communication device 317, an external I / F 316, and the like, and transmits / receives various data to / from the server 10 via the network N.

  The imaging unit 62 is realized by a command from the CPU 311, the imaging unit 301, the image processing unit 304, and the like, and captures an omnidirectional image and generates image data. That is, the imaging unit 62 generates image data including frame data for each imaging interval, for example, by capturing images at regular intervals or irregularly.

  The sound output unit 63 is realized by a command from the CPU 311, a sound output device 319, and the like, and outputs a sound such as a buzzer sound.

  In the present embodiment, as described above, the image data is described as moving image data, but is not limited thereto, and may be still image data. In this case, for example, the imaging unit 62 may generate a plurality of image data by capturing images at predetermined time intervals while the pedestrian 200 is walking.

  Then, the imaging unit 62 stores image data including time-series frame data in the storage unit 6000. Note that the imaging unit 62 stores a plurality of time-series image data in the storage unit 6000 when image data of a still image is generated.

  The storage / reading unit 69 is realized by any one or more of instructions from the CPU 311, ROM 312, SRAM 313, and DRAM 314. The storage / reading unit 69 stores various data in the storage unit 6000 and reads various data stored in the storage unit 6000. Perform the process.

<About walking of pedestrian 200>
Here, how the pedestrian 200A and the pedestrian 200B walk around the omnidirectional image capturing device 60 will be described. FIG. 7 is a diagram for explaining an example of walking of the pedestrian 200.

  The pedestrian 200A wears the acceleration measuring device 50A on the waist or the like. Similarly, the pedestrian 200B wears the acceleration measuring device 50B on the waist or the like. The place where the acceleration measuring device 50 is attached is preferably a place close to the center of gravity of the pedestrian 200, but may be attached to the chest, back, torso, etc. in addition to the waist. The pedestrian 200 may be equipped with a plurality of acceleration measuring devices 50.

At this time, as shown in FIG. 7 (a), the pedestrian 200A along from the point P ₁ the line L, walks to pivot around the omnidirectional imaging apparatus 60. Similarly, as shown in FIG. 7 (a), the pedestrian 200B along from the point P ₂ on the line L, walks to pivot around the omnidirectional imaging apparatus 60.

  Thus, when the pedestrian 200A and the pedestrian 200B walk along the line L, the omnidirectional image capturing device 60 can simultaneously capture the walking motions of the pedestrian 200A and the pedestrian 200B. Therefore, the instructor and the like can evaluate the walking image when the pedestrian 200A and the pedestrian 200B are walking while turning by checking the display image data.

  Note that a marker or the like may be arranged on the line L in order to make it easy to compare walking motions between the plurality of pedestrians 200 or to recognize a route on which the pedestrian 200 itself walks. Thereby, a leader etc. can grasp | ascertain easily the position of each pedestrian 200 with respect to the omnidirectional image imaging device 60, when confirming display image data.

  In addition, while each pedestrian 200 is walking along the line L, the acceleration measurement device 50 worn by each pedestrian 200 measures the acceleration of each pedestrian 200. Assuming that the relative position of the pedestrian 200 and the acceleration measuring device 50 is fixed, the traveling direction (front) of the pedestrian 200 is the x direction, the left and right direction is the y direction, and the vertical direction is the z direction.

  Here, the pedestrian 200 does not have to walk the entire circular line L, and may walk more than a distance appropriate for the evaluation of the walking motion. In addition, the pedestrian 200 may walk the circular line L one or more times.

Further, as shown in FIG. 7 (b), along the same point P ₁ the line L, first, the pedestrian 200A has started walking ahead, followed by pedestrians 200B starts to walk Also good.

FIG. 7B shows a case where the pedestrian 200 starts walking from the point P ₁ and walks along the line L including the straight line portion S. In this case, when the pedestrian 200 walks along the line L, since the pedestrian 200 makes a right turn or a left turn, the omnidirectional image capturing device 60 can capture the pedestrian 200 when making a right turn or a left turn.

  Further, since the pedestrian 200 reciprocates along the straight line portion S, the omnidirectional image capturing device 60 includes the pedestrian 200 walking from behind, the pedestrian 200 performing a 180 ° change of direction, and the omnidirectional image. A pedestrian 200 approaching the imaging device 60 from the front can be imaged.

  Note that the line L on which the pedestrian 200 walks is not limited to the above, and may be a figure 8 or the like, for example. That is, the line L should just be a shape preferable in order to evaluate walking motion. 7 shows a case where the pedestrian 200A and the pedestrian 200B walk while turning clockwise. For example, while the pedestrian 200A walks while turning clockwise, the pedestrian 200B You may walk while turning counterclockwise.

<Operation procedure>
≪Overall operation≫
FIG. 8 is a sequence diagram illustrating an example of the overall operation of the image display system 100. FIG. 8 is a sequence diagram when the server 10 issues a measurement start request and an imaging start request to the acceleration measuring device 50 and the omnidirectional image capturing device 60, respectively.

  First, when starting measurement of walking motion, a leader or the like inputs an operation for starting measurement of walking motion to the server 10. Then, the operation input reception part 12 of the server 10 receives operation of a leader etc. (step S101).

  Next, when the synchronization input unit 13 of the server 10 receives an operation of a leader or the like by the operation input receiving unit 12, a measurement start request is sent to each of the acceleration measuring device 50A and the acceleration measuring device 50B via the communication unit 11. Transmit (step S102 and step S103).

  When the communication unit 51 receives the measurement start request, the sound output unit 53 of the acceleration measurement device 50A and the acceleration measurement device 50B outputs a buzzer sound or the like (step S104 and step S105). Thereby, the pedestrian 200A wearing the acceleration measuring device 50A can know that the walking may be started. Similarly, the pedestrian 200B wearing the acceleration measuring device 50B can know that the walking may be started.

  The acceleration measuring device 50 notifies the pedestrian 200 of the start of walking by, for example, a voice message such as “Please start walking”, vibration by the vibration function, display of a message on the display 214, blinking of the display 214, or the like. May be.

  Further, the acceleration measuring device 50A and the acceleration measuring device 50B do not need to output a buzzer sound or the like at the same time. For example, when the pedestrian 200A starts walking before the pedestrian 200B, the acceleration measuring device 50A outputs a buzzer sound first, and then the acceleration measuring device 50B outputs a buzzer sound or the like. Also good.

  Next, the synchronization instruction unit 13 of the server 10 transmits an imaging start request to the omnidirectional image imaging device 60 via the communication unit 11 (step S106).

  When the communication unit 61 receives the imaging start request, the sound output unit 63 of the omnidirectional image capturing apparatus 60 outputs a buzzer sound or the like (step S107). Thereby, similarly to the above, the pedestrian 200 can know that the walking may be started.

  Note that the omnidirectional image capturing device 60 may notify the pedestrian 200 of the start of walking by means of a voice message, vibration, or the like, similar to steps S104 and S105 described above.

  Further, since the omnidirectional image capturing device 60 may enter the field of view of the pedestrian 200, for example, a patrol lamp or the like may be turned on. Thereby, not only the pedestrian 200 but also the surrounding leaders can know that the pedestrian 200 may start walking.

  Note that any one of notifications by the buzzer in steps S104, S105, and S107 may be performed.

  Next, the acceleration detector 52 of the acceleration measuring device 50A and the acceleration measuring device 50B starts measuring the acceleration, respectively (step S108 and step S109). Then, the acceleration detectors 52 of the acceleration measuring device 50A and the acceleration measuring device 50B respectively measure the acceleration generated while the pedestrian 200 is walking, and each of the measurement data including the measurement values obtained in time series is obtained. The data is stored in the storage unit 5000.

  Further, the imaging unit 62 of the omnidirectional image imaging device 60 starts imaging (step S110). Then, the imaging unit 62 of the omnidirectional image imaging device 60 stores image data obtained by imaging the walking pedestrian 200 in the storage unit 6000.

  As described above, the acceleration measuring device 50 and the omnidirectional image capturing device 60 can synchronize the measurement data measurement start and the image data imaging start almost simultaneously with an instruction from the server 10.

  Note that the order of step S102, step S103, and step S106 is random. The server 10 may transmit the measurement start request and the imaging start request to the acceleration measuring device 50 and the omnidirectional image capturing device 60, respectively, at the same time.

  Next, for example, when a leader or the like determines that the pedestrian 200 has sufficiently walked, the leader or the like inputs an operation to end the measurement of the walking motion to the server 10. Then, the operation input reception part 12 of the server 10 receives operation of a leader etc. (step S111).

  Next, when the synchronization instruction unit 13 of the server 10 receives an operation of a leader or the like by the operation input reception unit 12, a measurement end request is sent to each of the acceleration measurement device 50A and the acceleration measurement device 50B via the communication unit 11. Transmit (step S112 and step S113).

  And the acceleration detection part 52 of 50 A of acceleration measuring apparatuses and the acceleration measuring apparatus 50B will respectively complete | finish the measurement of acceleration, if the measurement end request | requirement is received by the communication part 51. FIG. Note that, similarly to step S104 and step S105, the end of measurement may be notified to the pedestrian 200 or the like with a buzzer or the like.

  Next, the synchronization instruction unit 13 of the server 10 transmits an imaging end request to the omnidirectional image imaging device 60 via the communication unit 11 (step S114). Then, when the imaging unit 62 of the omnidirectional image imaging device 60 receives the imaging end request through the communication unit 61, the imaging unit 62 ends imaging. Note that, similarly to step S107, the end of imaging may be notified to the pedestrian 200 or the like with a buzzer or the like.

  In this way, the acceleration measuring device 50 and the omnidirectional image capturing device 60 can synchronize the measurement data measurement end and the image data image capturing end almost simultaneously in response to an instruction from the server 10.

  In addition, the order of step S112, step S113, and step S114 is random. The server 10 may transmit a measurement end request and an imaging end request to the acceleration measuring device 50 and the omnidirectional image capturing device 60, respectively, at the same time.

  Next, the communication unit 51 of the acceleration measuring device 50A and the acceleration measuring device 50B transmits the measurement data A and the measurement data B stored in the storage unit 5000 to the server 10 (step S115 and step S116). Further, the communication unit 61 of the omnidirectional image capturing apparatus 60 transmits the image data stored in the storage unit 6000 to the server 10 (step S117).

  Next, when the acceleration data processing unit 14 of the server 10 receives the measurement data A, the measurement data B, and the image data through the communication unit 11, the measurement data A and the measurement data B are stored in the measurement data DB 1001, and the image data is converted into the image data. Each of them is stored in the DB 1002. Then, the acceleration data processing unit 14 processes the measurement data A and measurement data B, which are discrete data, into continuous data represented by, for example, a line graph (step S118). Such processing may be performed by the display device 30.

  Next, the display image creation unit 15 of the server 10 creates display image data based on the image data received from the omnidirectional image capturing device 60 (step S119). That is, for example, the display image creating unit 15 displays, for each frame data included in the image data, display frame data in which the frame data is associated with the pedestrian ID and position of the pedestrian 200 reflected in the frame data. create. Then, the display image creation unit 15 creates display image data so that the created display frame data is included. The pedestrian ID is identification information for identifying each pedestrian 200, and the same pedestrian ID is assigned to the same pedestrian 200.

  Thereby, when the several pedestrian 200 is reflected in the image imaged with the omnidirectional image imaging device 60, for example, in the display device 30, it displays for every pedestrian 200 according to selection of a leader etc. Can be switched. For this reason, it becomes easy for a leader etc. to perform confirmation of the walking motion of each pedestrian 200, comparison of walking motion between each pedestrian 200, etc. Details of the process in step S119 will be described later.

  Next, the image adjustment unit 16 of the server 10 performs a process of synchronizing the measurement data A and the time axis of the display image data, and the measurement data B and the time axis of the display image data, respectively (step S120). That is, the image adjustment unit 16 associates each measurement value included in the measurement data A, each display frame data included in the display image data, and the pedestrian ID of the pedestrian 200A. Similarly, the image adjustment unit 16 associates each measurement value included in the measurement data B, each display frame data included in the display image data, and the pedestrian ID of the pedestrian 200B.

  Here, the measurement data and the image data are acquired at almost the same time, but the number of measurement data (number of measurement values) is different from the number of display image data (number of display frame data). There are many cases. For this reason, the image adjustment unit 16 synchronizes the time axis by associating the measurement data with the display image data. Since the measurement start time and the imaging start time are almost the same, and the measurement end time and the imaging end time are almost the same, the image adjustment unit 16 synchronizes the time axis with the ratio of the number of measurement data and display image data. Can do.

  For example, when the number of measurement data is 200 and the number of display image data is 400, two display frame data included in the display image data may be associated with one measurement value included in the measurement data. Alternatively, the ratio of the measurement time of each measurement value with respect to the measurement time and the ratio of the imaging time of each display frame data with respect to the imaging time may be calculated, and the display frame data with the closest ratio may be associated with the measurement value. . Alternatively, the measurement value and the display frame data in which the elapsed time from the start of measurement and the elapsed time from the start of imaging are substantially the same may be associated with each other.

  The association between the measurement data and the pedestrian ID may be performed by, for example, an instructor or the like, and the measurement data and the pedestrian ID may be associated with each other based on the speed and representative color of each pedestrian 200 described later. Correlation may be performed.

  As will be described later, when the absolute time of the measurement start time, the measurement end time, the imaging start time, and the imaging end time is obtained, the image adjustment unit 16 performs the measurement data and the display image as follows. Data may be associated. That is, the image adjustment unit 16 extracts the measurement value and the display frame data in the overlapping portion between the time from the measurement start time to the measurement end time and the time from the imaging start time to the imaging end time, and the ratio similar to the above What is necessary is just to associate by calculation etc.

  Next, the communication unit 11 of the server 10 transmits the measurement data A, the measurement data B, and the display image data to the display device 30 (Steps S121 to S123).

  Specifically, for example, a leader or the like requests measurement data and display image data from the server 10 via the display device 30 or the like. Or a leader etc. may perform operation which requests | requires measurement data and display image data with respect to the server 10, and the operation input reception part 12 of the server 10 may receive this operation. Alternatively, the display device 30 may inquire about the presence of measurement data and display image data by polling or the like, and the server 10 may transmit the measurement data and display image data to the display device 30 in response to the inquiry.

  Next, when the measurement data and display image data are received by the communication unit 31, the display control unit 32 of the display device 30 displays the measurement data and display image data on the display 108 on the same time axis (step S124). A process for displaying the measurement data and the display image data will be described later.

≪Modification of overall operation (part 1) ≫
Here, a modified example (part 1) of the overall operation of the image display system 100 will be described. FIG. 9 is a sequence diagram illustrating a modification example (No. 1) of the overall operation of the image display system 100. In the modified example (part 1) shown in FIG. 9, the trigger for starting measurement is different from that in FIG. Note that steps S203 to S211 and steps S214 to S226 in FIG. 9 are the same as steps S102 to S110 and steps S112 to S124 in FIG.

  First, when starting measurement of walking motion, a leader or the like inputs an operation for starting measurement of walking motion to the display device 30. Then, the operation input receiving unit 33 of the display device 30 receives an operation of a leader or the like (step S201).

  Next, the communication part 31 of the display apparatus 30 will transmit a measurement start request | requirement to the server 10, if operation of a leader etc. is received by the operation input reception part 33 (step S202). The display device 30 can communicate with the server 10 by, for example, HTTP (Hypertext Transfer Protocol). The measurement start request is a request for at least one of acceleration measurement start and pedestrian 200 imaging start.

  For example, when the leader or the like determines that the pedestrian 200 has sufficiently walked, the leader or the like inputs an operation to end the measurement of the walking motion to the display device 30. Then, the operation input receiving unit 33 of the display device 30 receives an operation of a leader or the like (step S212).

  Next, the communication part 31 of the display apparatus 30 will transmit a measurement end request | requirement to the server 10, if operation of a leader etc. is received by the operation input reception part 33 (step S213). The measurement end request is a request for at least one of the acceleration measurement end and the pedestrian 200 imaging end.

  As described above, in the modified example (No. 1) of the overall operation illustrated in FIG. 9, the leader or the like requests the server 10 to start and end the measurement from the display device 30 without operating the server 10. Can do. For this reason, when a leader or the like is operating the display device 30, the leader or the like can start measurement without having to move to the server 10.

<< Modified example of overall operation (2) >>
Next, a modification (No. 2) of the overall operation of the image display system 100 will be described. FIG. 10 is a sequence diagram illustrating a modification (No. 2) of the overall operation of the image display system 100. In the modification (No. 2) shown in FIG. 10, the start of measurement is triggered by the start of walking by the pedestrian 200. The pedestrian 200 starts and ends walking, for example, according to instructions from a leader or the like. Alternatively, walking is started by starting walking from the start position on the line L, going around the line L and returning to the starting position. Note that steps S305 to S311 and steps S316 to S326 in FIG. 10 are the same as steps S104 to S110 and S114 to S124 in FIG.

  First, the acceleration detection unit 52 of the acceleration measuring device 50A and the acceleration measuring device 50B detects that the pedestrian 200A and the pedestrian 200B have started walking based on the acceleration, respectively (step S301 and step S302).

  For example, the acceleration detection unit 52 can detect that the pedestrian 200 has started walking by detecting an acceleration equal to or greater than a threshold value in one or more of the x direction, the y direction, and the z direction. In addition, for example, the acceleration detection unit 52 may detect that the pedestrian 200 has started walking because a change in position per unit time obtained by integrating acceleration twice is equal to or greater than a threshold value. . In addition to these, the acceleration detection unit 52 is configured so that the pedestrian 200 walks based on the angular velocity detected by the gyro sensor 206, the position detected by the GPS receiver 213, the direction detected by the acceleration / direction sensor 205, and the like. May be detected.

  And the acceleration detection part 52 of 50 A of acceleration measuring apparatuses and the acceleration measuring apparatus 50B each starts the measurement of acceleration by detecting the start of walking of the pedestrian 200A and the pedestrian 200B, respectively.

  Next, the communication unit 51 of the acceleration measuring device 50A and the acceleration measuring device 50B transmits a walk start notification to the server 10 (step S303 and step S304).

  The walking start notification may be transmitted to the server 10 by at least one acceleration measuring device 50 (for example, the acceleration measuring device 50 that detects the start of walking first). However, in step S307, the server 10 may transmit an imaging start request to the omnidirectional image imaging device 60 when the walking start notification is received from all the acceleration measuring devices 50.

  The acceleration detector 52 of the acceleration measuring device 50A and the acceleration measuring device 50B detects that the pedestrian 200A and the pedestrian 200B have stopped walking based on the acceleration, respectively (step S312 and step S313).

  For example, the acceleration detection unit 52 can detect that the pedestrian 200 has stopped walking because no acceleration greater than or equal to the threshold is detected in one or more of the x direction, the y direction, and the z direction. Further, for example, the acceleration detection unit 52 may detect that the pedestrian 200 has stopped walking because the change in the position of the unit time obtained by integrating the acceleration twice becomes less than the threshold value. In addition to these, the acceleration detection unit 52 is configured so that the pedestrian 200 walks based on the angular velocity detected by the gyro sensor 206, the position detected by the GPS receiver 213, the direction detected by the acceleration / direction sensor 205, and the like. It may be detected that has stopped.

  Then, the acceleration detectors 52 of the acceleration measuring device 50A and the acceleration measuring device 50B detect the stop of walking of the pedestrian 200A and the pedestrian 200B, respectively, thereby ending the measurement of acceleration.

  The communication units 51 of the acceleration measuring device 50A and the acceleration measuring device 50B each transmit a walking end notification to the server 10 (steps S314 and S315).

  As described above, in the modified example (No. 2) of the overall operation illustrated in FIG. 10, the acceleration measuring device 50 performs the walking operation of the pedestrian 200 even if the leader or the like does not perform an operation such as measurement start or measurement end. By detecting the start and end of the measurement, the measurement can be started and ended.

<< Modified example of overall operation (part 3) >>
Next, a modification (No. 3) of the overall operation of the image display system 100 will be described. FIG. 11 is a sequence diagram for explaining a modification (No. 3) of the overall operation of the image display system 100. In the modification (No. 3) shown in FIG. 11, the measurement data and the image data are synchronized using the absolute time. Note that steps S401 to S405 and steps S408 to S409 in FIG. 11 are the same as steps S101 to S105 and steps S106 to S107 in FIG. Further, Step S411 to Step S416, Step S419, and Step S421 to Step S430 in FIG. 11 are the same as Step S108 to Step S113, Step S114, and Step S115 to Step S124 in FIG. Is omitted.

  When receiving the measurement start request, the communication units 51 of the acceleration measurement device 50A and the acceleration measurement device 50B respectively transmit a start time request to the server 10 (steps S406 and S407). Then, the synchronization instruction unit 13 of the server 10 sends the absolute time (measurement start time) when the acceleration measurement device 50A and the acceleration measurement device 50B start measurement via the communication unit 11 to the acceleration measurement device 50A and the acceleration measurement device 50B. Reply to each. Thereby, the acceleration measuring device 50 can obtain the measurement start time.

  Further, when receiving the imaging start request, the communication unit 61 of the omnidirectional image capturing apparatus 60 transmits a start time request to the server 10 (step S410). Then, the synchronization instruction unit 13 of the server 10 returns the absolute time (imaging start time) when the omnidirectional image capturing device 60 starts capturing via the communication unit 11. Thereby, the omnidirectional image imaging device 60 can obtain the imaging start time.

  When the communication unit 51 of the acceleration measuring device 50A and the acceleration measuring device 50B receives the measurement end request, it transmits an end time request to the server 10 (step S417 and step S418). Then, the synchronization instruction unit 13 of the server 10 transmits the absolute time (measurement end time) at which the acceleration measurement device 50A and the acceleration measurement device 50B finish measurement via the communication unit 11 to the acceleration measurement device 50A and the acceleration measurement device 50B. Reply to each. Thereby, the acceleration measuring apparatus 50 can obtain the measurement end time.

  When receiving the imaging end request, the communication unit 61 of the omnidirectional image capturing apparatus 60 transmits an end time request to the server 10 (step S420). Then, the synchronization instruction unit 13 of the server 10 returns the absolute time (imaging end time) when the omnidirectional image capturing device 60 has completed the imaging via the communication unit 11. Thereby, the omnidirectional image imaging device 60 can obtain the imaging end time.

  As described above, in the modified example (No. 3) of the overall operation illustrated in FIG. 11, the server 10 returns the measurement start time and the measurement end time to the acceleration measuring device 50, and sets the imaging start time and the imaging end time. It returns to the omnidirectional image capturing device 60. For this reason, in the modified example (part 3), the server 10 can associate the measurement data and the display image data on the basis of the absolute time in step S426. Therefore, for example, even if a time lag or the like occurs between the start of acceleration measurement and the start of imaging for some reason, or between the end of measurement of acceleration and the end of imaging, the server 10 determines the time between measurement data and display image data. Axis can be synchronized.

<< Modified example of overall operation (Part 4) >>
Next, a modification (No. 4) of the overall operation of the image display system 100 will be described. FIG. 12 is a sequence diagram illustrating a modification (No. 4) of the overall operation of the image display system 100. In the modification (No. 4) illustrated in FIG. 12, one acceleration measurement device 50 among the plurality of acceleration measurement devices 50 transmits a measurement start request to the other acceleration measurement device 50, and an omnidirectional image. An imaging start request is transmitted to the imaging device 60. In response to a request from the other acceleration measuring device 50 and the omnidirectional image capturing device 60, the one acceleration measuring device 50 returns the absolute time.

  Note that steps S501 and S508 to S509 in FIG. 12 are the same as steps S101 and S104 to S105 in FIG. 12 are the same as Step S107 to Step S111 and Step S118 to Step S124 of FIG. 8, respectively, and thus description thereof is omitted.

  The synchronization instruction unit 13 of the server 10 receives a measurement start request via the communication unit 11 when the operation input reception unit 12 receives an operation of a leader or the like (operation for starting measurement of walking motion). 50A is transmitted (step S502). Note that the communication unit 11 transmits a measurement start request to one of the plurality of acceleration measuring devices 50 set in advance. Therefore, for example, the communication unit 11 may transmit a measurement start request to the acceleration measurement device 50B. Hereinafter, a case where the communication unit 11 of the server 10 transmits a measurement start request to the acceleration measuring device 50A will be described.

  When receiving the measurement start request, the communication unit 51 of the acceleration measuring device 50A transmits the measurement start request to the acceleration measuring device 50B and transmits the imaging start request to the omnidirectional image capturing device 60 (steps S503 and S504). ).

  When receiving the measurement start request, the communication unit 51 of the acceleration measurement device 50B transmits a start time request to the acceleration measurement device 50A (step S505). Further, when receiving the imaging start request, the communication unit 61 of the omnidirectional image imaging device 60 transmits a start time request to the acceleration measuring device 50A (step S506).

  Next, when receiving the start time request from the acceleration measurement device 50B and the omnidirectional image capturing device 60, the communication unit 51 of the acceleration measurement device 50A transmits the start time request to the server 10 (step S507). Then, the synchronization instruction unit 13 of the server 10 returns the measurement start time via the communication unit 11. Then, when receiving the measurement start time, the communication unit 51 of the acceleration measurement device 50A returns the measurement start time to the acceleration measurement device 50B and returns the imaging start time to the omnidirectional image imaging device 60.

  The acceleration measuring device 50A may return the same time as the measurement start time received from the server 10 to the omnidirectional image imaging device 60 as the imaging start time.

  In step S505 and step S506, even if the time (current time) held by the acceleration measurement device 50A is returned to the acceleration measurement device 50B and the omnidirectional image imaging device 60 as the measurement start time and imaging start time. good. The acceleration measuring device 50 starts measuring acceleration from the current time, and the omnidirectional imaging device 60 starts imaging from the imaging start time. Therefore, the measurement start time and the imaging start time can be synchronized. .

  The synchronization instruction unit 13 of the server 10 receives an operation of an instructor or the like (operation for ending the measurement of walking motion) by the operation input receiving unit 12, and sends a measurement end request via the communication unit 11 to the acceleration measuring device. It transmits to 50A (step S515).

  When receiving the measurement end request, the communication unit 51 of the acceleration measuring device 50A transmits the measurement end request to the acceleration measuring device 50B and transmits the imaging end request to the omnidirectional image capturing device 60 (steps S516 and S517). ).

  When receiving the measurement end request, the communication unit 51 of the acceleration measuring device 50B transmits an end time request to the acceleration measuring device 50A (step S518). Further, when receiving the imaging end request, the communication unit 61 of the omnidirectional image imaging device 60 transmits an end time request to the acceleration measuring device 50A (step S519).

  Next, when receiving the end time request from the acceleration measuring device 50B and the omnidirectional image capturing device 60, the communication unit 51 of the acceleration measuring device 50A transmits the end time request to the server 10 (step S520). Then, the synchronization instruction unit 13 of the server 10 returns the measurement end time via the communication unit 11. Then, when receiving the measurement end time, the communication unit 51 of the acceleration measurement device 50A returns the measurement end time to the acceleration measurement device 50B and returns the imaging end time to the omnidirectional image imaging device 60.

  The acceleration measurement device 50A may return the same time as the measurement end time received from the server 10 to the omnidirectional image imaging device 60 as the imaging end time.

  In step S518 and step S519, the time (current time) held by the acceleration measurement device 50A is returned to the acceleration measurement device 60A and the omnidirectional image imaging device 60 as the measurement end time and imaging end time. good. The acceleration measuring device 50 finishes measuring the acceleration at the current time, and the omnidirectional imaging device 60 finishes imaging at the imaging end time, so that the measurement end time and the imaging end time can be synchronized. .

  The communication unit 51 of the acceleration measuring device 50B transmits the measurement data B stored in the storage unit 5000 to the acceleration measuring device 50A (step S521). Further, the communication unit 61 of the omnidirectional image capturing apparatus 60 transmits the image data stored in the storage unit 6000 to the acceleration measuring apparatus 50A (step S522).

  The communication unit 51 of the acceleration measuring device 50A includes the measurement data A stored in the storage unit 500, the measurement data B received from the acceleration measuring device 50B, and the image data received from the omnidirectional image capturing device 60. Is transmitted to the server 10 (step S523).

  As described above, in the modified example (No. 4) of the overall operation illustrated in FIG. 12, one acceleration measuring device 50 returns the absolute time to the other acceleration measuring device 50 and the omnidirectional image capturing device 60. , Relay measurement data and image data. For this reason, for example, when no other acceleration measuring device 50 or omnidirectional image capturing device 60 is connected to the network N, one acceleration measuring device 50 can perform near field communication such as Bluetooth (registered trademark). Can be used to relay data to and from the server 10.

<< Modified example of overall operation (part 5) >>
Next, a modification (No. 5) of the overall operation of the image display system 100 will be described. FIG. 13 is a sequence diagram for explaining a modification (No. 5) of the overall operation of the image display system 100. In the modified example (No. 5) shown in FIG. 13, the omnidirectional image capturing device 60 relays the measurement data measured by each acceleration measuring device 50 and transmits it to the server 10.

  Note that steps S601 and S605 to S606 in FIG. 13 are the same as steps S101 and S104 to S105 in FIG. Further, Steps S607 to S611 and Steps S618 to S624 in FIG. 13 are the same as Steps S107 to S111 and Steps S118 to S124 in FIG.

  The synchronization instruction unit 13 of the server 10 receives an operation of an instructor or the like (operation for starting the measurement of walking motion) by the operation input receiving unit 12, and sends an imaging start request via the communication unit 11. It transmits to the image pick-up device 60 (step S602).

  Next, when receiving the imaging start request, the communication unit 61 of the omnidirectional image imaging device 60 transmits a measurement start request to each of the acceleration measuring device 50A and the acceleration measuring device 50B (steps S603 and S604). As described above, when the omnidirectional image capturing apparatus 60 receives the imaging start request from the server 10, the omnidirectional image capturing apparatus 60 transmits the measurement start request to the acceleration measuring apparatus 50.

  The synchronization instruction unit 13 of the server 10 receives an imaging end request via the communication unit 11 when the operation input reception unit 12 receives an operation of a leader or the like (operation for ending the measurement of walking motion). It transmits to the image pick-up device 60 (step S612).

  Next, when receiving the imaging end request, the communication unit 61 of the omnidirectional image imaging device 60 transmits a measurement end request to each of the acceleration measuring device 50A and the acceleration measuring device 50B (steps S613 and S614). As described above, when the omnidirectional image capturing apparatus 60 receives the imaging end request from the server 10, the omnidirectional image capturing apparatus 60 transmits the measurement end request to the acceleration measuring apparatus 50.

  Next, the communication unit 51 of the acceleration measuring device 50A and the acceleration measuring device 50B transmits the measurement data A and the measurement data B stored in the storage unit 5000 to the omnidirectional image pickup device 60 (step S615 and step S615). S616).

  Then, the communication unit 61 of the omnidirectional imaging device 60 includes the image data stored in the storage unit 6000, the measurement data A received from the acceleration measurement device 50A, and the measurement data B received from the acceleration measurement device 50B. Is transmitted to the server 10 (step S617).

  As described above, in the modified example (No. 5) of the overall operation illustrated in FIG. 13, the omnidirectional image capturing device 60 relays and transmits a measurement start request to the acceleration measuring device 50. For this reason, for example, when the acceleration measuring device 50 is not connected to the network N, the omnidirectional image capturing device 60 relays data to and from the server 10 using short-range wireless communication such as Bluetooth. It can be carried out.

≪Display image creation process≫
Next, display image creation processing in step S119 of FIG. 8 will be described. FIG. 14 is a flowchart illustrating an example of the display image creation process. Hereafter, N frame data is included in the image data generated by the omnidirectional image capturing device 60 every time t = t _n (n = 1, 2,..., N). It shall be assumed. Frame data “f00n” (n = 1, 2,..., N) is assigned to the frame data at time t = t _n (n = 1, 2,..., N), respectively. To do. Note that Δt _n = t _n −t _n−1 (n = 2, 3,..., N) is an imaging interval.

First, the display image creation unit 15 acquires frame data at time t = t ₁ from image data (step S701). That is, the display image creation unit 15 acquires frame data of the frame number “f001” from the image data.

  Next, the display image creating unit 15 specifies the pedestrian 200 shown in the frame data acquired in step S701, and calculates the position of the specified pedestrian 200 (step S702). The position calculated here is, for example, the xy coordinates when the x-axis is defined in the horizontal direction and the y-axis is defined in the vertical direction of the frame image.

  Here, the display image creation unit 15 may identify the pedestrian 200 by recognizing the face of the pedestrian 200, for example. However, it is not restricted to this, For example, the display image preparation part 15 may identify the pedestrian 200 by recognizing the leg | foot etc. of the pedestrian 200.

  The display image creation unit 15 may set the position of the pedestrian 200 at the center or the center of gravity of the area recognized as a face or the area recognized as a foot, for example.

  Next, the display image creation unit 15 assigns a pedestrian ID to each pedestrian 200 identified in step S702 (step S703).

Here, for example, when two pedestrians 200 are specified in step S702 described above, the display image creating unit 15 may assign pedestrian IDs “m001” and “m002” in the specified order. Thus, when the display image creation unit 15 specifies each pedestrian 200 in the frame data at time t = t ₁ , the display image creation unit 15 assigns a pedestrian ID that identifies each pedestrian 200.

Next, the display image creation unit 15 associates the position of the pedestrian 200 identified in step S702 and the pedestrian ID of the pedestrian 200 with the frame data of the frame number “f001”, thereby displaying the display. Frame data is created (step S704). As a result, display frame data at time t = t ₁ is created.

That is, for example, two pedestrians 200 are specified in the above step S702, the pedestrian IDs are “m001” and “m002”, and the respective positions are “x ₁₁ , y ₁₁ ” and “x ₂₁ , y”. ₂₁ ”. In this case, the display image creating unit 15 adds the pedestrian ID “m001” and the position “x ₁₁ , y ₁₁ ”, the pedestrian ID “m002” and the position “x ₂₁ , y” to the frame data of the frame number “f001”. ₂₁ ”.

Next, the display image creation unit 15 sets time t = t _n (n = 2) (step S705).

Next, the display image generation unit 15 acquires the frame data at time t = _{t n} from the image data (step S706). In other words, the display image creating unit 15 acquires frame data of the frame number “f00n” from the image data.

  Next, the display image creating unit 15 specifies the pedestrian 200 shown in the frame data acquired in step S706 and calculates the position of the specified pedestrian 200 (step S707).

  Note that the display image creation unit 15 may identify the pedestrian 200 and calculate the position of the pedestrian 200 by the same method as in step S702 described above. Further, the display image creating unit 15 identifies the pedestrian 200 by identifying a rectangular area in which the pedestrian 200 is shown, for example, and uses the center of the rectangular area and the position of the center of gravity as the position of the pedestrian 200. Also good.

  Here, the specification of the rectangular area in which the pedestrian 200 is shown will be described with reference to FIG. FIG. 15 is a diagram schematically illustrating an example of specifying a rectangular area in which a pedestrian 200 is shown.

First, in FIG. 15 (a), the frame image 181 frame data at time t = _{t n} is displayed, the frame image 182 that time _{t =} t _n-1 frame data is displayed (i.e., the previous frame image 181 Frame image) is displayed. For example, when the frame rate is 30 fps, there is an interval of about 33 (milliseconds) between the frame image 181 and the immediately preceding frame image 182.

  The frame image 182 may be a frame image that is several frames before the frame image 181 instead of the frame image 181 and the immediately preceding frame image 182.

  The display image creation unit 15 calculates the difference between the frame image 181 and the immediately preceding frame image 182. The difference means that the pixel values of the pixels at the same position in the frame image 181 and the frame image 182 are subtracted. Note that even if the exact same subject is shown (even if there is no moving object), the difference may not be zero due to the blinking of the fluorescent light, the movement of a distant person, the change of the shadow, etc. Consider zero.

  FIG. 15B is an example of the difference image. In FIG. 15B, a pixel whose difference is greater than or equal to a threshold is replaced with a white pixel. Therefore, when the frame image 181 and the immediately preceding frame image 182 are compared, it can be said that there is a high possibility that the pedestrian 200 exists so as to overlap the white pixels.

  Next, as illustrated in FIG. 15C, the display image creating unit 15 performs pixel expansion processing on white pixels a plurality of times. The pixel expansion process is a process of replacing 8 pixels in contact with white pixels (or 4 pixels adjacent vertically and horizontally) with white pixels. Since the pixel in contact with the white pixel is traced by a plurality of pixel expansion processes, the white pixel appears to expand each time the number of times increases.

  The multiple times is a predetermined number such as 8 times or 10 times, and a suitable value is experimentally determined. Alternatively, the pixel expansion process may be stopped when the area of the white pixel exceeds a certain value.

  By executing the pixel expansion process a plurality of times, the white pixels that are scattered immediately after the difference between the frame image 181 and the immediately preceding frame image 182 is calculated become a block having a size. In the case of the pedestrian 200, since the difference image is vertically long, the white pixel is enlarged in the vertical direction by the pixel expansion process, and a slightly larger size than the moving object (a range slightly larger than the pedestrian 200) can be specified.

  Next, as shown in FIG. 15D, a circumscribed rectangle of a region where white pixels are continuous is specified as a rectangular region 601. Thereby, the rectangular area | region 601 in which the pedestrian 200 is reflected is specified.

Returning to FIG. The display image creation unit 15 assigns a pedestrian ID to each pedestrian 200 identified in step S707 based on the pedestrian ID and position associated with the frame data at time t = t _n−1. (Step S708).

That is, for each pedestrian 200 identified in step S707, the display image creation unit 15 sets the pedestrian ID associated with a position within a predetermined range from the position of the pedestrian 200 at time t = t _{n. -1 from} the frame data. Then, the display image creating unit 15 sets the acquired pedestrian ID as the pedestrian ID of the pedestrian 200.

For example, the position “x _1n−1 ” associated with the frame data at time t = t _n−1 within a predetermined range from the position “x _1n , y _1n ” of the pedestrian 200 identified in step S707 above. , Y _1n-1 ”. In this case, the display image creation unit 15 sets the pedestrian ID “m001” associated with the position “x _1n−1 , y _1n−1 ” in the frame data at time t = t _n−1 to the above step. This is given to the pedestrian 200 identified in S707.

As a result, the display image creation unit 15 makes the pedestrian 200 close to the same pedestrian 200 between the frame data at time t = t _{n−1 and} the frame data at time t = t _n. As a thing, the same pedestrian ID can be given. That is, the display image creation unit 15 can assign a pedestrian ID that identifies each pedestrian 200 that moves around the omnidirectional image capturing device 60.

For example, when a plurality of positions associated with the frame data at time t = t _n−1 are included within a predetermined range from the position of the pedestrian 200 identified in step S707 above, a plurality of positions Of these, a pedestrian ID associated with the closest position may be given.

Further, the display image creation unit 15 does not include a position associated with the frame data at time t = t _n−1 within a predetermined range from the position of the pedestrian 200 identified in step S707 above. A different pedestrian ID is assigned to the identified pedestrian 200. This is because the display image creation unit 15 may specify a new pedestrian 200, for example, in step S707 described above.

Next, the display image creating unit 15 associates the position of the pedestrian 200 identified in step S707 and the pedestrian ID of the pedestrian 200 with the frame data of the frame number “f00n” to display the pedestrian 200. Frame data is created (step S709). Thus, the display frame data at time t = t _n is created.

That is, for example, two pedestrians 200 are specified in the above step S707, the respective pedestrian IDs are “m001” and “m002”, and the respective positions are “x _1n , y _1n ” and “x _2n , y”. _2n ". In this case, the display image creating unit 15 adds the pedestrian ID “m001” and the position “x _1n , y _1n ”, the pedestrian ID “m002” and the position “x _2n , y” to the frame data of the frame number “f00n”. _2n ".

  Next, the display image creation unit 15 determines whether n is less than N (step S710). If n is less than N, the display image creation unit 15 adds 1 to n (step S711), and returns to step S706.

  In this way, the display image creation unit 15 creates display frame data from each frame data included in the image data for n = 2 to N.

  Here, the position of the pedestrian 200 associated with each display frame data will be described with reference to FIG. FIG. 16 is a diagram illustrating an example of the position of the pedestrian 200 associated with each display frame data.

Figure 16 shows a pedestrian ID and location associated with the display frame data fn at each time t = t _n. That is, as shown in FIG. 16, the display frame data f1 at time t = t ₁ includes the pedestrian ID “m001” and the position “x ₁₁ , y ₁₁ ”, the pedestrian ID “m002”, and the position “x _21. , Y ₂₁ ”.

Similarly, the display frame data f2 at time t = t ₂ includes a pedestrian ID “m001” and a position “x ₁₂ , y ₁₂ ”, a pedestrian ID “m002” and a position “x ₂₂ , y ₂₂ ”. It is associated. Similarly, the display frame data fN at time t = t _N includes a pedestrian ID “m001” and a position “x _1N , y _1N ”, a pedestrian ID “m002” and a position “x _2N , y _2N ”. Are associated with each other.

  Thus, each display frame data fn is associated with the pedestrian ID of the pedestrian 200 shown in the display frame data fn and the position of the pedestrian 200. Thereby, in the image display system 100 which concerns on this embodiment, a leader etc. can evaluate the operation | movement of each pedestrian 200 who walks the circumference | surroundings of the omnidirectional image pick-up device 60 separately or comparing. become.

On the other hand, when n is not less than N, the display image creation unit 15 creates display image data including display frame data at each time t = t _n (n = 1, 2,..., N) (step S712). ).

  For example, when the display image data is reproduced by streaming or the like on the display device 30, the server 10 may transmit the display frame data to the display device 30 after the process of step S <b> 709 is executed.

<< Other examples of display image creation processing (part 1) >>
Next, another example (part 1) of the display image creation process in step S119 of FIG. 8 will be described. FIG. 17 is a flowchart for explaining another example (part 1) of the display image creation process. In another example (part 1) of the display image creation process shown in FIG. 17, a pedestrian ID is assigned to each pedestrian 200 based on the speed of each pedestrian 200 shown in the display frame data. It is. As a result, the display image creation unit 15 can assign the same pedestrian ID to the same pedestrian 200 with higher accuracy.

  Note that step S801, step S805 to step S806, and step S810 to step S812 in FIG. 17 are the same as step S701, step S705 to step S706, and step S710 to step S712 in FIG. Omitted.

Subsequent to step S801, the display image creation unit 15 specifies the pedestrian 200 shown in the frame data acquired in step S801, and calculates the position and speed of the specified pedestrian 200 (step S802). Here, the speed of the pedestrian 200 may be calculated by, for example, integrating the acceleration of the identified pedestrian 200 at time t = t ₁ .

  Next, the display image creation unit 15 assigns a pedestrian ID to each pedestrian 200 identified in step S802 (step S803). Note that the display image creation unit 15 may assign a pedestrian ID to each pedestrian 200 in the same manner as in step S703 in FIG.

Next, the display image creating unit 15 associates the position and speed of the pedestrian 200 identified in step S802 and the pedestrian ID of the pedestrian 200 with the frame data of the frame number “f001”. Display frame data is created (step S804). As a result, display frame data at time t = t ₁ is created.

  Subsequent to step S806, the display image creating unit 15 specifies the pedestrian 200 shown in the frame data acquired in step S806, and calculates the position and speed of the specified pedestrian 200 (step S807). The display image creating unit 15 may identify the pedestrian 200 and calculate the position and speed of the pedestrian 200 by the same method as in step S802 above.

Next, the display image creation unit 15 adds to each pedestrian 200 identified in step S807 based on the pedestrian ID, position, and speed associated with the frame data at time t = t _n−1. A pedestrian ID is assigned (step S808).

That is, the display image creating unit 15 associates each identified pedestrian 200 with a position within a predetermined range from the position of the pedestrian 200 and with a speed within a predetermined range from the speed of the pedestrian 200. Pedestrian ID is obtained from the frame data at time t = t _n−1 . Then, the display image creating unit 15 sets the acquired pedestrian ID as the pedestrian ID of the pedestrian 200.

Thus, the display image generation unit 15, for example, in the frame data at time t = t _n, even when different pedestrian 200 is captured near, based on the speed of the pedestrian 200, the same pedestrian 200 can be given the same pedestrian ID. Specifically, the display image creating unit 15 performs the same walking for the same pedestrian 200 even when the pedestrian 200A passes the pedestrian 200B or when the pedestrian 200A and the pedestrian 200B pass each other. Person ID can be given.

Next, the display image creation unit 15 associates the position and speed of the pedestrian 200 identified in step S807 and the pedestrian ID of the pedestrian 200 with the frame data of the frame number “f00n”. Display frame data is created (step S809). Thus, the display frame data at time t = t _n is created.

  Here, the position of the pedestrian 200 associated with each display frame data will be described with reference to FIG. FIG. 18 is a diagram illustrating another example (part 1) of the position of the pedestrian associated with each display frame data.

Figure 18 is a pedestrian ID associated with the display frame data fn at each time t = t _n, it indicates a position, and speed. That is, as shown in FIG. 16, the display frame data f1 at time t = t ₁ includes the pedestrian ID “m001”, the position “x ₁₁ , y ₁₁ ”, the speed “v ₁₁ ”, and the pedestrian ID “ m002 ”, position“ x ₂₁ , y ₂₁ ”, and speed“ v ₂₁ ”are associated with each other.

Similarly, the display frame data f2 at time t = t ₂ includes a pedestrian ID “m001”, a position “x ₁₂ , y ₁₂ ”, a speed “v ₁₂ ”, a pedestrian ID “m002”, and a position “x”. ₂₂ , y ₂₂ ”and speed“ v ₂₂ ”are associated with each other. Similarly, the display frame data fN at time t = t _N includes a pedestrian ID “m001”, a position “x _1N , y _1N ”, a speed “v _1N ”, a pedestrian ID “m002”, a position “X _2N , y _2N ” and speed “v _2N ” are associated with each other.

  Thus, each display frame data fn is associated with the pedestrian ID of the pedestrian 200 reflected in the display frame data fn and the position and speed of the pedestrian 200. Thereby, in the image display system 100 according to the present embodiment, the same pedestrian ID can be given to the same pedestrian 200 with higher accuracy than the display image creation processing of FIG. become.

≪Other example of display image creation processing (part 2) ≫
Next, another example (part 2) of the display image creation process in step S119 of FIG. 8 will be described. FIG. 19 is a flowchart for explaining another example (part 2) of the display image creation process. In another example (part 2) of the display image creation process shown in FIG. 19, the pedestrian ID is assigned to each pedestrian 200 based on the color (representative color) of each pedestrian 200 shown in the display frame data. Is given. As a result, the display image creation unit 15 can assign the same pedestrian ID to the same pedestrian 200 with higher accuracy.

  Note that step S901, step S905 to step S906, and step S910 to step S912 in FIG. 19 are the same as step S701, step S705 to step S706, and step S710 to step S712 in FIG. Omitted.

  Subsequent to step S901, the display image creation unit 15 specifies the pedestrian 200 shown in the frame data acquired in step S901, and calculates the position and representative color of the specified pedestrian 200 (step S902). Here, the representative color of the pedestrian 200 may be the color of the pixel at the specified position of the pedestrian 200, for example. However, the present invention is not limited to this, and the representative color of the pedestrian 200 may be, for example, an average color of surrounding pixels including the specified position of the pedestrian 200 (for example, surrounding 8 pixels).

  Next, the display image creation unit 15 assigns a pedestrian ID to each pedestrian 200 identified in step S902 (step S903). Note that the display image creation unit 15 may assign a pedestrian ID to each pedestrian 200 in the same manner as in step S703 in FIG.

Next, the display image creation unit 15 associates the position and representative color of the pedestrian 200 specified in step S902 above and the pedestrian ID of the pedestrian 200 with the frame data of the frame number “f001”. Thus, display frame data is created (step S904). As a result, display frame data at time t = t ₁ is created.

  Subsequent to step S906, the display image creating unit 15 specifies the pedestrian 200 in the frame data acquired in step S906, and calculates the position and representative color of the specified pedestrian 200 (step S907). The display image creating unit 15 may identify the pedestrian 200 and calculate the position and representative color of the pedestrian 200 by the same method as in step S902 above.

Next, the display image creation unit 15 determines each pedestrian 200 identified in step S907 above based on the pedestrian ID, position, and representative color associated with the frame data at time t = t _n−1. Is given a pedestrian ID (step S908).

That is, the display image creation unit 15 associates each identified pedestrian 200 with a representative color that is close to the representative color of the pedestrian 200 and a position within a predetermined range from the position of the pedestrian 200. The pedestrian ID is acquired from the frame data at time t = t _n−1 . Then, the display image creating unit 15 sets the acquired pedestrian ID as the pedestrian ID of the pedestrian 200.

Note that the closeness of representative colors corresponds to the specified representative color of the pedestrian 200 and frame data at time t = t _{n−1 in} a predetermined color space (for example, RGB color space, CMY color space, etc.). This means that the distance from the attached representative color is not more than a predetermined threshold.

Thus, the display image generation unit 15, for example, in the frame data at time t = t _n, even when different pedestrian 200 is captured near, based on the representative color of the pedestrian 200, the same walking The person 200 can be given the same pedestrian ID. Therefore, in the image display system 100 according to the present embodiment, for example, by causing each pedestrian 200 to wear clothes of different colors, the same pedestrian ID is given to the same pedestrian 200 with higher accuracy. Will be able to.

Next, the display image creation unit 15 associates the position and representative color of the pedestrian 200 identified in step S907 and the pedestrian ID of the pedestrian 200 with the frame data of the frame number “f00n”. Display frame data is created (step S909). Thus, the display frame data at time t = t _n is created.

  Here, the position of the pedestrian 200 associated with each display frame data will be described with reference to FIG. FIG. 20 is a diagram for explaining another example (part 2) of the position of the pedestrian associated with each display frame data.

Figure 20 is a pedestrian ID associated with the display frame data fn at each time t = t _n, it indicates a position, and the representative color. That is, as shown in FIG. 20, the display frame data f1 at time t = t ₁ includes a pedestrian ID “m001”, a position “x ₁₁ , y ₁₁ ”, a representative color “# FF2398”, and a pedestrian ID. “M002”, position “x ₂₁ , y ₂₁ ”, and representative color “# 87AC75” are associated with each other.

Similarly, the display frame data f2 at time t = t ₂ includes a pedestrian ID “m001”, a position “x ₁₂ , y ₁₂ ”, a representative color “# FF2398”, a pedestrian ID “m002”, and a position “ x ₂₂ , y ₂₂ ”and the representative color“ # FF87AC74 ”are associated with each other. Similarly, the display frame data fN of time t = _{t N,} pedestrians ID "m001" position and _{"x 1N, y 1N",} and the representative color "# FF2397", pedestrians ID "m002" The position “x _2N , y _2N ” and the representative color “# 87AC75” are associated with each other.

  In this way, each display frame data fn is associated with the pedestrian ID of the pedestrian 200 reflected in the display frame data fn, the position of the pedestrian 200, and the representative color. Thereby, in the image display system 100 according to the present embodiment, the same pedestrian ID can be assigned to the same pedestrian 200 with higher accuracy than in FIG.

<Display image data and measurement data>
Next, display examples of display image data and measurement data will be described. FIG. 21 is a diagram illustrating an example of display image data and measurement data displayed on the display device 30.

  A data display screen 501 shown in FIG. 21 mainly includes an image data column 502, a selection column 503, and a measurement data column 504. The display control unit 32 displays the display image data in the image data column 502 and displays the measurement data in the measurement data column 504.

  At this time, the display control unit 32 takes out the imaging range in which the pedestrian 200 having the pedestrian ID selected in the selection field 503 is captured from the display image data and displays it in the image data field 502. This process will be described later.

  First, the image data column 502 will be described. When a leader or the like selects a pedestrian ID in the selection field 503 and presses the button 505, the operation input receiving unit 33 receives the pressing operation. And the display control part 32 reproduces | regenerates as a moving image at the same reproduction speed as a frame rate (fps), after taking out the imaging range which the pedestrian 200 of the selected pedestrian ID is reflected from display image data. Note that high-speed playback, low-speed playback, and the like may be possible.

  When a leader or the like presses the button 505 again, the operation input receiving unit 33 receives the pressing operation. Then, the display control unit 32 stops the reproduction of the display image data. In this way, a leader or the like can display display image data at an arbitrary timing. Note that not only the display image data but also the sound collected together with the image data A and the image data B may be output.

  When a leader or the like selects another pedestrian ID from the selection field 503 during reproduction of the display image data, the operation input reception unit 33 receives the selection operation. And the display control part 32 reproduces | regenerates as a moving image, after taking out the imaging range in which the selected pedestrian ID is reflected from display image data.

  When the display image data is a still image, when a leader or the like presses the button 505, the display control unit 32 displays the next display image data in the image data column 502. Similarly, the display control unit 32 can display the previous display image data in the image data field 502 by a button operation. The display control unit 32 may display the display image data in the image data column 502 one after another at a predetermined time interval.

  Next, the measurement data column 504 will be described. In FIG. 21, the accelerations in the x direction, the y direction, and the z direction of the pedestrian 200 with the pedestrian ID selected in the selection field 503 are displayed in time series. In FIG. 21, acceleration in any one or more directions may be displayed by an operation of a leader or the like among the x direction, the y direction, or the z direction. The measurement data is continuously displayed. For example, when the display image data is a still image, the measurement data is measured by the acceleration measuring device 50 at the timing when the omnidirectional image capturing device 60 captures the image. Acceleration may be displayed.

  The display control unit 32 scales the measurement data so that all the measurement values included in the measurement data are displayed in the measurement data column 504, and displays them in the measurement data column 504. Note that the display control unit 32 may display measurement values for a certain amount of time in the measurement data column 504 and switch them at predetermined intervals.

  As shown in FIG. 21A, the display control unit 32 displays a bar 506 indicating the measurement value associated with the pedestrian ID of the pedestrian 200 displayed in the image data field 502 in the measurement data field 504. Let For example, when the measurement values for the whole or a certain time are displayed in the measurement data column 504 and the display image data is reproduced, the display control unit 32 displays the pedestrian 200 pedestrian displayed in the frame image. A bar 506 is superimposed on the measurement value associated with the ID.

  Therefore, the display control unit 32 moves the bar 506 to the right in the measurement data column 504 as time elapses or as the reproduction of the moving image proceeds. When a leader or the like moves the bar 506, the operation input accepting unit 33 accepts the operation, and the display control unit 32 displays a frame image of the display frame data associated with the measurement value indicated by the moved bar 506. May be displayed in the image data field 502.

  Also, as shown in FIG. 18B, an image data field 502A for displaying a pedestrian 200A with a pedestrian ID “m001” and an image data field 502B for displaying a pedestrian 200B with a pedestrian ID “m002” are displayed. May be. At this time, similarly, a measurement data column 504A for displaying the measurement data A of the pedestrian 200A with the pedestrian ID “m001” and a measurement data column 504B for displaying the measurement data B of the pedestrian 200B with the pedestrian ID “m002” are displayed. Is displayed.

  Thereby, the leaders can easily evaluate the walking behavior of the plurality of pedestrians 200 without comparing the pedestrian IDs in the selection field 503.

≪About evaluation of measurement data≫
Here, when the measurement data is acceleration, a peak appears periodically in a general way of walking, and the height of the peak does not vary much. On the other hand, if the walking function deteriorates due to age, injury, or the like, the peak of acceleration does not appear periodically or the height of the peak tends to fluctuate.

  For this reason, the instructor or the like may be able to evaluate the walking motion only by looking at the display image data, but can further support the measurement data that the walking motion is not appropriate. In addition, a leader or the like can identify a place where the measurement data is disturbed and can check the display image data. Although the display image data is displayed only for a frame image at a certain moment, since the measurement data has a list property, an instructor or the like can quickly display a frame image on which an inappropriate walking motion is displayed based on the measurement data. You can search by

  In the present embodiment, the acceleration is described as an example. However, the angular velocity detected by the gyro sensor 206, the position detected by the GPS receiver 213, the azimuth detected by the acceleration / direction sensor 205, etc. It is also possible to evaluate whether the walking motion is appropriate.

  For example, the angular velocity detected by the gyro sensor 206 indicates the speed of change of the yaw angle, pitch angle, and roll angle when the pedestrian 200 walks. In this case as well, as with acceleration, it is known that peaks appear periodically in a general way of walking and the height of the peak does not vary much.

  Further, since the acceleration is obtained by differentiating the position twice, the position detected by the GPS receiver 213 and the like can be used as information equivalent to the acceleration. In the case of turning walking, the azimuth constantly changes, but acceleration of the change in azimuth can be obtained by differentiating the azimuth twice. In this case as well, the peak period and height are a guide for evaluation. In addition, in these measurement data, even if the peak is periodic and the height fluctuation is small, if the peak is extremely large, there may be an improvement in walking motion. Can confirm the display image data.

  The display control unit 32 of the display device 30 may calculate and display the area between the acceleration and the axis of acceleration = 0. For example, when the pedestrian 200 walking in the shape of figure 8 is turning, the y direction is the acceleration in the left-right direction of the pedestrian 200, so that the acceleration is an area on the positive side and an area on the negative side. Indicates left and right blurring with respect to the line L. If the pedestrian 200 walks on the line L, the area on the positive side and the area on the negative side are almost equal. Therefore, when the area on the positive side is significantly different from the area on the negative side, it can be seen that the balance between the left leg and the right leg of the pedestrian 200 is poor.

  Thus, a leader or the like can obtain a lot of information for evaluating the walking motion of the pedestrian 200 by looking at the measurement data and the display image data whose time axes are synchronized.

<Rotation of spherical image>
In the display image data, an image of 360 ° around the omnidirectional image capturing device 60 is shown. For this reason, the display control unit 32 displays only the range in which the pedestrian 200 of the selected pedestrian ID is shown in the image data field 502 according to the pedestrian ID selected by the instructor or the like. Such processing is expressed as rotation of the omnidirectional image, trimming, extraction or extraction of the pedestrian 200, or the like. The rotation of the omnidirectional image refers to determining the range displayed in the image data field 502 by rotating the omnidirectional image in the horizontal direction (in the longitude direction).

  FIG. 22 is a flowchart illustrating an example of a process in which the image rotation unit 34 specifies a range in which the pedestrian 200 with the selected pedestrian ID is shown and displays the range in the image data field 502. The process of FIG. 22 is started, for example, when the display device 30 receives display image data and the display image data is reproduced.

  First, the image rotation part 34 acquires the position matched with pedestrian ID selected by the leader etc. from display frame data. Then, the image rotation unit 34 rotates the frame image so that the acquired position becomes the center of the display range (S1001). That is, when the display frame data included in the display image data is switched, the image rotation unit 34 acquires a position associated with the selected pedestrian ID so that the position becomes the center of the display range. Trimming is performed by rotating the frame image.

  As a result, the pedestrian 200 with the pedestrian ID selected by the instructor or the like can be captured in the approximate center of the frame image.

  The display control unit 32 displays the frame image of the display range trimmed by the image rotation unit 34 in the image data field 502 (step S1002). For this reason, a leader etc. can evaluate walking action of the pedestrian 200, without searching for the pedestrian 200 from the omnidirectional image by which the range of 360 degrees was imaged.

  Note that the server 10 or the omnidirectional image capturing device 60 may have the function of the image rotation unit 34. In this case, the communication load and communication time when display image data is transmitted from the server 10 or the omnidirectional image capturing device 60 to the display device 30 can be reduced.

<Other System Configurations of Image Display System 100>
Here, another system configuration of the image display system 100 will be described. FIG. 23 is a diagram illustrating another example of the system configuration of the image display system 100.

  FIG. 23A shows another example of the system configuration of the image display system 100. In the image display system 100 illustrated in FIG. 23A, the server 10 includes a display 108. In the configuration shown in FIG. 23A, the server 10 has various functions of the display device 30. That is, the server 10 displays the measurement data and the display image data by rotating the frame image in addition to the creation of the display image data and the synchronization of the time axes of the measurement data and the image data.

  FIG. 23B is another example of the system configuration of the image display system 100. The image display system 100 illustrated in FIG. 23B does not include the server 10 and the display device 30. In the configuration shown in FIG. 23B, the acceleration measuring device 50A or the acceleration measuring device 50B has various functions of the server 10 and the display device 30.

  Here, the overall operation in the image display system 100 shown in FIG. 23B will be described. FIG. 24 is a sequence diagram illustrating an example of the overall operation of the image display system 100 when the acceleration measuring device 50A displays measurement data and display image data.

  First, when starting measurement of walking motion, a leader or a pedestrian 200A or the like inputs an operation for starting measurement of walking motion to the acceleration measuring device 50A. Then, the acceleration measuring device 50A accepts an operation of a leader or a pedestrian 200A (step S1101).

  Next, the communication unit 51 of the acceleration measuring device 50A transmits a measurement start request to the acceleration measuring device 50B and transmits an imaging start request to the omnidirectional image capturing device 60 (steps S1102 and S1103).

  Next, the acceleration detector 52 of the acceleration measuring device 50A and the acceleration measuring device 50B starts measuring acceleration (steps S1104 and S1105). In addition, the imaging unit 62 of the omnidirectional image imaging device 60 starts imaging the pedestrian 200 (step S1106).

  Next, when the walking of the pedestrian 200 is finished, the instructor or the pedestrian 200A inputs an operation for ending the measurement of the walking motion to the acceleration measuring device 50A. Then, the acceleration measuring device 50A receives an operation of a leader or a pedestrian 200A (step S1107).

  Next, the communication unit 51 of the acceleration measuring device 50A transmits a measurement end request to the acceleration measuring device 50B and transmits an imaging end request to the omnidirectional image capturing device 60 (steps S1108 and S1109).

  Next, when the communication unit 51 receives a measurement end request, the acceleration detection unit 52 of the acceleration measuring device 50B ends the acceleration measurement. Then, the communication unit 51 of the acceleration measuring device 50B transmits the measurement data B stored in the storage unit 5000 to the acceleration measuring device 50A (step S1110).

  In addition, when the imaging unit 62 of the omnidirectional image imaging device 60 receives the imaging end request from the communication unit 61, the imaging unit 62 ends the imaging. Then, the communication unit 61 of the omnidirectional image capturing device 60 transmits the image data stored in the storage unit 6000 to the acceleration measuring device 50A (step S1111).

  Next, the acceleration measuring device 50A processes the measurement data as in step S118 of FIG. 8 (step S1112).

  Next, the acceleration measuring device 50A creates display image data as in step S119 of FIG. 8 (step S1113).

  Next, the acceleration measuring device 50A synchronizes the time axes of the measurement data and the image data as in step S120 of FIG. 8 (step S1114).

  Next, the acceleration measuring device 50A displays measurement data and display image data on the display 214 of the own device (step S1115). In addition, 50 A of acceleration measuring apparatuses may transmit measurement data and display image data to terminals, such as a leader (server 10 grade | etc.,) (Step S1116). In this case, a terminal such as a leader displays measurement data and display image data (step S1117).

  Therefore, according to the configuration shown in FIG. 23B, the system configuration of the image display system 100 can be simplified. The pedestrian 200 walks while wearing the acceleration measuring device 50. After the end of walking, the pedestrian 200 can receive guidance from the instructor by passing the acceleration measuring device 50 to the instructor or the like.

<Other application examples>
The best mode for carrying out the present invention has been described above with reference to the embodiments. However, the present invention is not limited to these embodiments, and various modifications can be made without departing from the scope of the present invention. And substitutions can be added.

  For example, in the present embodiment, the measurement data and the display image data are displayed after the measurement data is measured and the image data is captured. However, the measurement data and the display image data may be displayed in real time. In this case, the displayed measurement data and display image data are taken at substantially the same time.

  In addition, the configuration example of FIG. 6 and the like shown in the above embodiment is mainly used for easy understanding of the processing of the server 10, the display device 30, the omnidirectional image capturing device 60, and the acceleration measuring device 50. It is divided according to the function. For example, a plurality of servers 10 may exist, and a plurality of servers 10 may cooperate to perform the processing of this embodiment. Further, the storage unit 1000 of the server 10 does not need to be included in the server 10, and the storage unit 1000 only needs to be in a place where the server 10 can read and write data in the storage unit 1000.

  In FIG. 6, the functions of the server 10, the display device 30, the omnidirectional image capturing device 60, and the acceleration measuring device 50 have been described by being divided into several processing units. However, the present invention is not limited by the way of dividing or the name of each processing unit. The processing of the server 10, the display device 30, the omnidirectional image capturing device 60, and the acceleration measuring device 50 can be divided into more processing units depending on the processing content. Moreover, it can also divide | segment so that one process unit may contain many processes.

DESCRIPTION OF SYMBOLS 10 Server 15 Display image creation part 16 Image adjustment part 30 Display apparatus 32 Display control part 33 Operation input reception part 34 Image rotation part 50 Acceleration measuring apparatus 60 Omnispherical image imaging device 100 Image display system

JP 2010-172394 A

Claims (19)

An image display system comprising: an information processing device; a wide-angle imaging device capable of imaging a surrounding area in the horizontal direction; and one or more measuring devices that respectively measure movements of one or more moving objects.
Image data generated by imaging the one or more objects by the wide-angle imaging device, and one or more measurement data generated by measuring the operations of the one or more objects by the one or more measurement devices, respectively. Acquisition means for acquiring;
A first association unit that identifies each of the one or more target positions in the image data, and associates the identified position with the image data;
A second association means for associating each of the one or more measurement data with the image data;
Display means for displaying on the display device the image data and the one or more measurement data associated with the image data based on the position associated with the image data;
An image display system. The wide-angle imaging device
Imaging the one or more objects moving around the wide-angle imaging device while changing direction or changing direction;
The first association means includes
Specifying the positions of the one or more objects in the image data obtained by imaging the one or more objects moving while changing directions or changing directions, and associating the specified positions with the image data;
The second association means includes
2. The image display system according to claim 1, wherein each of the one or more measurement data is associated with image data obtained by capturing the one or more objects that progress while changing directions or changing directions. The information processing apparatus includes:
Requesting each of the one or more measurement devices to start measurement and end measurement, and to instruct the wide-angle imaging device to start imaging and end imaging,
The second association means includes
Based on the measurement start of each of the one or more measurement data and the imaging start of the image data, the measurement end of each of the one or more measurement data and the imaging end of the image data, The image display system according to claim 1, wherein the image data is associated with the image data. The display device
Receiving means for receiving at least one of measurement start of the target motion or imaging start of the target;
Notification means for notifying the information processing apparatus of at least one of the measurement start or the imaging start received by the reception means;
The image display system according to claim 3, comprising: The measurement data includes one or more measurement values obtained by measuring the operation of the object,
The measuring device is
When the measurement value equal to or greater than a predetermined threshold is detected, the measurement data is started and the measurement means that requests the wide-angle imaging device to start imaging,
The wide-angle imaging device
Upon receiving a request to start imaging by the measuring unit, the imaging unit starts imaging the target and starts generating the image data.
The image display system according to claim 1, comprising: The measuring means includes
When the measurement value less than a predetermined threshold is detected, the generation of the measurement data is terminated, and the wide-angle imaging device is requested to end imaging.
The imaging means includes
The image display system according to claim 5, wherein upon receiving a request to end imaging by the measuring unit, imaging of the target is terminated and generation of the image data is terminated. The wide-angle imaging device
The image display system according to claim 1, further comprising: a first notification unit that notifies the start of imaging of the target. The measuring device is
The image display system according to any one of claims 1 to 7, further comprising a second notification unit that notifies that measurement of the movement of the target is started. The first association means includes
For each frame image data included in the image data, specify the position of the one or more objects in the frame image data, and associate the specified position with the frame image data.
The second association means includes
Associating a measurement value included in each of the one or more measurement data with frame image data included in the image data;
The display means includes
Based on the position associated with the frame image data, the frame image data included in the image data and the one or more measurement values associated with the frame image data are displayed on the display device. The image display system according to any one of claims 1 to 8. The display means includes
The image display system according to claim 9, wherein in the frame image data, a region in a predetermined range including the position associated with the frame image data is displayed on the display device. The first association means includes
For each frame image data included in the image data, the position of the one or more objects in the frame image data is specified, and the specified position and identification information for identifying the object are associated with the frame image data. With
The display means includes
In accordance with the designation of the target identification information, the frame image data has a predetermined range including the position associated with the designated identification information among the positions associated with the frame image data. The image display system according to claim 10, wherein an area is displayed on the display device. The first association means includes
For each frame image data included in the image data, the identification information of the object is specified based on the speed of the one or more objects in the frame image data, and the specified identification information and the position are determined as the frame image. The image display system according to any one of claims 9 to 11, which is associated with data. The first association means includes
For each frame image data included in the image data, the identification information of the object is specified based on the color of the one or more objects in the frame image data, and the specified identification information and the position are determined as the frame image. The image display system according to any one of claims 9 to 11, which is associated with data. An information processing apparatus capable of communicating with a wide-angle imaging device capable of imaging a surrounding area in the horizontal direction and one or more measurement devices that measure the movements of one or more moving objects,
Image data generated by imaging the one or more objects by the wide-angle imaging device, and one or more measurement data generated by measuring the operations of the one or more objects by the one or more measurement devices, respectively. Acquisition means for acquiring;
A first association unit that identifies each of the one or more target positions in the image data, and associates the identified position with the image data;
A second association means for associating each of the one or more measurement data with the image data;
Display means for displaying on the display device the image data and the one or more measurement data associated with the image data based on the position associated with the image data;
An information processing apparatus. An information processing apparatus capable of communicating with a wide-angle imaging device capable of imaging a surrounding area in the horizontal direction and a measurement device that measures the movement of a moving first object and generates first measurement data,
Measuring means for measuring the movement of the moving second object and generating second measurement data;
Acquisition means for acquiring image data generated by the wide-angle imaging device imaging the first object and the second object, and the first measurement data;
The first position of the first object and the second position of the second object in the image data are specified, and the specified first position and the second position are associated with the image data. First association means;
Second association means for associating each of the first measurement data and the second measurement data with the image data;
Based on the first position and / or the second position associated with the image data, the image data and the first measurement data and / or the first position associated with the image data. Display means for displaying two measurement data on a display device;
An information processing apparatus. In a wide-angle imaging device capable of imaging a surrounding area in the horizontal direction, one or more measuring devices each measuring the motion of one or more moving objects, and an imaging device capable of communicating with an information processing device,
Imaging means for generating image data generated by imaging the one or more objects by the wide-angle imaging device;
Obtaining means for obtaining one or more measurement data generated by the one or more measurement devices respectively measuring the movements of the one or more objects;
Transmitting means for transmitting the image data and the one or more measurement data to the information processing apparatus;
An imaging apparatus having An information processing device capable of communicating with a wide-angle imaging device capable of imaging a surrounding area in the horizontal direction and one or more measuring devices that respectively measure movements of one or more moving objects;
Image data generated by imaging the one or more objects by the wide-angle imaging device, and one or more measurement data generated by measuring the operations of the one or more objects by the one or more measurement devices, respectively. Acquisition means to acquire,
First associating means for identifying the positions of the one or more objects in the image data, and associating the identified positions with the image data;
A second association means for associating each of the one or more measurement data with the image data;
Display means for causing the display device to display the image data and the one or more measurement data associated with the image data based on the position associated with the image data;
Program to function as. An information processing device capable of communicating with a wide-angle imaging device capable of imaging the periphery of the horizontal direction and a measurement device that measures the movement of the moving first object and generates first measurement data,
Measuring means for measuring the movement of the moving second object and generating second measurement data;
Acquisition means for acquiring image data generated by the wide-angle imaging device imaging the first object and the second object, and the first measurement data;
The first position of the first object and the second position of the second object in the image data are specified, and the specified first position and the second position are associated with the image data. First association means;
A second association means for associating each of the first measurement data and the second measurement data with the image data;
Based on the first position and / or the second position associated with the image data, the image data and the first measurement data and / or the first position associated with the image data. Display means for displaying two measurement data on a display device;
Program to function as. A wide-angle imaging device capable of imaging a surrounding area in the horizontal direction, wherein one or more measuring devices that respectively measure movements of one or more moving objects, and an imaging device that can communicate with an information processing device,
Imaging means for generating image data generated by the wide-angle imaging device imaging the one or more objects;
Obtaining means for obtaining one or more measurement data generated by the one or more measurement devices respectively measuring the movements of the one or more objects;
Transmitting means for transmitting the image data and the one or more measurement data to the information processing apparatus;
Program to function as.