JP6729043B2 - Information processing device and information processing system - Google Patents

Description

The present invention relates to an information processing device and an information processing system.

A technique for identifying the position of a person included in an image is disclosed. For example, there are known a technique of specifying the center of gravity of a person included in an image as the position of the person, and a technique of specifying the head as the position of the person. Attempts have also been made to improve the accuracy of identifying the position of a person.

For example, Patent Document 1 discloses that position information detected by a detection device is corrected according to a position where a flow line is missing.

Here, the position of a person who does not feel discomfort when visually recognized varies depending on the situation of the person included in the image. However, conventionally, a specific position such as the center of gravity or the head of the person included in the image is specified as the position of the person. Further, in the technique of Patent Document 1, the correction is performed according to the flow line of the person, but it is not possible to specify a position that does not cause a sense of discomfort when visually recognized as the position of the person as the position of the person. That is, conventionally, it has been difficult to improve the accuracy of identifying the position of a person in an image.

The present invention has been made in view of the above, and an object thereof is to identify the position of a person in an image with high accuracy.

In order to solve the above-mentioned problems and achieve the object, the present invention provides a detection unit that detects a human region included in an image, a posture determination unit that determines the posture of a person shown in the human region, and the posture. A position identifying unit that identifies the position of the person in the image based on the above, and a superposition determining unit that determines whether another object is superposed or non-superposed on the person shown in the person region in the image. A moving body discriminating unit which discriminates whether the other object is a moving body or a non-moving body . When it is determined that another object is superposed on the person shown in the person area, and the other object is a non-moving object, the position specifying unit determines a predetermined area in the person area. If the position is specified as the position of the person, and it is determined that another object is superimposed on the person shown in the person area, and that the other object is a moving body, the person is The position of the person in the image is specified based on the posture of the person shown in the area, and when it is determined that another object is not superposed on the person shown in the person area, the position is determined based on the posture. The position of the person in the image is specified.

According to the present invention, there is an effect that the position of a person in an image can be accurately specified.

FIG. 1 is a schematic diagram of an information processing system according to the present embodiment. FIG. 2 is an example of an external appearance schematic diagram of the image capturing apparatus. FIG. 3 is a diagram illustrating an example of the usage status of the image capturing apparatus. FIG. 4 is an explanatory diagram of an example of creating a spherical panoramic image. FIG. 5 is an explanatory diagram of an example of creating a spherical panoramic image. FIG. 6 is an explanatory diagram of the positional relationship between the spherical panoramic image and the predetermined area. FIG. 7 is an explanatory diagram of the positional relationship between the spherical panoramic image and the predetermined area. FIG. 8 is a schematic diagram showing the hardware configuration of the image capturing apparatus. FIG. 9 is a hardware configuration diagram of the server device and the information processing terminal. FIG. 10 is an explanatory diagram showing an example of the outline of the flow of information processing. FIG. 11 is a schematic diagram showing the functional configurations of the server device, the photographing device, the communication terminal, and the information processing terminal. FIG. 12 is a schematic diagram showing an example of the data structure of the first specific information and the second specific information. FIG. 13 is a schematic diagram showing an example of the data structure of the image management DB. FIG. 14 is an explanatory diagram of detection of a person area. FIG. 15 is an explanatory diagram showing an example of specifying the position of a person. FIG. 16 is an explanatory diagram illustrating an example of identifying the position of a person. FIG. 17 is an explanatory diagram showing an example of identifying the position of a person. FIG. 18 is a schematic diagram showing an example of the setting screen. FIG. 19 is a schematic diagram showing an example of a display image. FIG. 20 is a flowchart showing an example of the procedure of information processing. FIG. 21 is a flowchart showing an example of an information processing procedure. FIG. 22 is a schematic diagram showing the functional configurations of the server device, the photographing device, the communication terminal, and the information processing terminal. FIG. 23 is a schematic diagram showing an example of the data structure of the image management DB. FIG. 24 is an explanatory diagram of a display image. FIG. 25 is a flowchart showing an example of an information processing procedure. FIG. 26 is a flowchart showing an example of an information processing procedure.

Hereinafter, embodiments of the present invention will be described.

(First embodiment)
FIG. 1 is a schematic diagram of an information processing system 10 according to this embodiment.

The information processing system 10 includes a server device 12, a photographing device 14, and an information processing terminal 18. The server device 12, the imaging device 14, and the information processing terminal 18 are communicably connected to each other via a communication unit such as the network 20.

For the communication unit, for example, a short-range wireless technology, a wireless communication network using a mobile communication system, the Internet, or the like is used. Examples of the short-range wireless technology include Wi-Fi (Wireless Fidelity). Examples of the wireless communication network based on the mobile communication system include LTE (Long Term Evolution), NFC (Near Field Communication), and WiMAX (Worldwide interoperability for Microwave Access).

The image capturing device 14 captures an omnidirectional range and obtains an omnidirectional panoramic image. In the present embodiment, the photographing device 14 is connected to the network 20 via the communication terminal 15 and the wireless router 16.

The server device 12 is an example of an information processing device. The server device 12 identifies the position of the person in the image (details will be described later). In the present embodiment, an example will be described in which the target image for specifying the position of the person is a spherical panoramic image. The omnidirectional panoramic image is an image obtained by photographing in an omnidirectional (360°) range.

The image used by the server device 12 to specify the position of the person is not limited to the spherical panoramic image. The information processing terminal 18 receives the omnidirectional panoramic image managed by the server device 12 and displays the omnidirectional panoramic image.

Note that FIG. 1 illustrates a case where the number of each of the server device 12, the imaging device 14, and the information processing terminal 18 included in the information processing system 10 is one. However, the numbers of the server device 12, the imaging device 14, and the information processing terminal 18 may each be plural.

First, the photographing device 14 will be described.

The photographing device 14 is a photographing device that obtains a spherical panoramic image by photographing. The omnidirectional panoramic image is a panoramic image obtained by photographing in the omnidirectional range (360°).

FIG. 2 is an example of an external appearance schematic diagram of the photographing device 14. FIG. 2A is a side view of the photographing device 14. FIG. 2B is a side view of the imaging device 14 on the opposite side of FIG. 2A. FIG. 2C is a plan view of the photographing device 14.

As shown in FIG. 2A, the imaging device 14 has a size that, for example, a person can hold with one hand. The size of the photographing device 14 is not limited to such a size.

As shown in FIG. 2, a lens 14A is provided on the front side (one surface side) and a lens 14B is provided on the back side (other surface side) on the upper part of the photographing device 14. Each of the lens 14A and the lens 14B is a wide-angle lens having an angle of view of 180° or more. In the present embodiment, the lenses 14A and 14B will be described as being fish-eye lenses, which are one of wide-angle lenses. In the imaging device 14, an image is guided through each of the lens 14A and the lens 14B, and an image is formed on the image pickup element corresponding to each. Examples of the image pickup device include CCD and CMOS. Further, as shown in FIG. 2B, an operation unit 14C such as a shutter button may be provided on the front side of the photographing device 14.

Next, an example of the usage status of the imaging device 14 will be described. FIG. 3 is a diagram illustrating an example of the usage status of the image capturing apparatus 14. As shown in FIG. 3, when the user holds the image capturing device 14 in his/her hand and takes an image, all directions (360°) around the user are imaged. The images guided by the lens 14A and the lens 14B are detected by the respective image pickup elements, so that the subject around the user is imaged and two hemispherical images are obtained. A spherical panoramic image is created from these two hemispherical images. The image capturing device 14 may be installed on a support member installed on the ground or a wall. In this case, all directions around the installation position are photographed.

Next, the outline of the process from the hemispherical image captured by the image capturing device 14 to the creation of the spherical panoramic image will be described. 4 and 5 are explanatory diagrams of an example of creating the omnidirectional panoramic image 60. 4A is a hemispherical image captured by the image capturing device 14 (front side), FIG. 4B is a hemispherical image captured by the image capturing device 14 (rear side), and FIG. 4C is represented by the Mercator projection. It is a schematic diagram which shows an example of the omnidirectional panoramic image 60.

The spherical panoramic image 60 is a three-dimensional stereoscopic spherical image in which a pair of hemispherical images (see FIGS. 4A and 4B) are attached to the inner wall of a 360° spherical body. (See FIG. 5).

Here, as shown in FIG. 4A, since the lens 14A is a fisheye lens, the image obtained by this lens 14A is a curved hemispherical image (front side). Further, as shown in FIG. 4B, since the lens 14B is a fisheye lens, the image obtained by this lens 14B is a curved hemispherical image (rear side). Then, the hemispherical image (front side) and the hemispherical image (rear side) are combined by the photographing device 14 to create the omnidirectional panoramic image 60.

Then, in the present embodiment, the photographing device 14 sequentially transmits the spherical panoramic image 60 to the server device 12 each time the spherical panoramic image 60 is obtained by photographing (see FIG. 1).

In the present embodiment, it is assumed that the imaging device 14 synthesizes the hemispherical images into the spherical panoramic image 60 and then transmits the panoramic image 60 to the server device 12. In the present embodiment, the case where the image capturing apparatus 14 transmits the omnidirectional panoramic image 60 (FIG. 4C) represented by the Mercator projection method to the server apparatus 12 will be described as an example.

The server device 12 may receive the pair of hemispherical images from the image capturing device 14 and synthesize the hemispherical images to create the omnidirectional panoramic image 60. The image capturing device 14 may also transmit to the image capturing device 14 a three-dimensional spherical spherical panoramic image 60 (see FIG. 5) in which a hemispherical image is attached to the inner wall of the three-dimensional solid sphere. ..

In this case, the imaging device 14 attaches the pair of hemispherical images so as to cover the inner wall of the spherical surface by using the OpenGL ES (Open Graphics Library for Embedded Systems). As a result, the image capturing device 14 creates a spherical omnidirectional panoramic image 60 as shown in FIG. In this case, the three-dimensional stereoscopic spherical panoramic image 60 is represented as an image represented by the Mercator projection in which the center of the sphere is oriented. Note that OpenGL ES is a graphics library used to visualize 2D (2-Dimensions) and 3D (3-Dimensions) data.

Here, the omnidirectional panoramic image 60 is a 360° omnidirectional image attached so as to cover the inner wall of the spherical surface as described above. Therefore, at the time of display, it is preferable to display an image centered on the virtual viewpoint direction in all 360° directions. Further, in this case, it is preferable to display a part of the predetermined area of the omnidirectional panoramic image 60 as a flat image with little curvature centered on the viewpoint direction. As a result, it is possible to provide a display that does not give a sense of discomfort to humans.

6 and 7 are explanatory diagrams of the positional relationship between the omnidirectional spherical panoramic image 60, which is assumed to be a three-dimensional solid spherical shape, and the predetermined area S.

It is assumed that the spherical panoramic image 60 is a three-dimensional solid spherical (solid sphere CS) as shown in FIGS. 6 and 7. In this case, the virtual photographing device C is positioned at a position where the inner wall of the solid sphere CS can be photographed. The virtual imaging device C corresponds to the viewpoint position of the user who visually recognizes the spherical panoramic image 60 displayed as the three-dimensional solid sphere CS.

Then, a coordinate system with the center of the solid sphere CS as the origin is constructed. The predetermined area S is an area corresponding to a shooting angle of view when the virtual shooting device C shoots a predetermined viewpoint direction in the spherical spherical panoramic image 60 with a narrower viewing angle than the spherical panoramic image 60. Is.

The predetermined area S is specified by the viewpoint direction information. In the present embodiment, the viewpoint direction information is the three-dimensional coordinate information of one vertex CP among the four vertices of the rectangular predetermined area S in the omnidirectional panoramic image 60, which is assumed to be a three-dimensional stereoscopic spherical shape ( x, y, z) and the range (width and height (w, h)) of the predetermined area S.

The predetermined area S may be represented by the coordinates (x, y) and the shooting angle of view α. The coordinates (x, y) indicate the center position CP′ of the predetermined area S in the solid sphere CS when the position of the virtual imaging device C is the origin. The shooting angle of view α is specified by the shooting magnification (zoom) of the virtual shooting device C. The predetermined area information may further include shooting posture information indicating the posture of the virtual shooting device C. The shooting posture information of the virtual shooting device C is specified by the rotation angles (yaw, pitch, roll) of the three axes that form the coordinate system with the center of the solid sphere CS as the origin.

In the information processing system 10 of the present embodiment, the predetermined area S in the omnidirectional panoramic image 60 is displayed as the viewpoint direction on the information processing terminal 18 or the like. Therefore, the information processing system 10 has a configuration capable of displaying without giving a sense of discomfort to humans. The viewpoint direction can be changed by an operation instruction from a user who operates the information processing terminal 18. That is, the user inputs the instruction to change the viewpoint direction using the information processing terminal 18 to change the viewpoint direction in the omnidirectional panorama image 60 (that is, the area to be displayed on the display screen in the omnidirectional panorama image 60). Can be changed.

Next, the hardware configuration of the photographing device 14 will be described.

FIG. 8 is a schematic diagram showing the hardware configuration of the photographing device 14. The imaging device 14 includes an imaging unit 101, an image processing unit 104, an imaging control unit 105, a microphone 108, a sound processing unit 109, a CPU (Central Processing Unit) 111, a ROM (Read Only Memory) 112, and an SRAM (Static Random Access Memory). 113, a DRAM (Dynamic Random Access Memory) 114, an operation unit 115, a network I/F 116, a communication unit 117, an antenna 117a, and an electronic compass 118.

The image pickup unit 101 includes a lens 14A, a lens 14B, an image pickup element 13A, and an image pickup element 13B. The image sensor 13A is provided corresponding to the lens 14A. The image sensor 13B is provided corresponding to the lens 14B. The image pickup device 13A and the image pickup device 13B convert the optical images formed by the lenses 14A and 14B into image data of electric signals and output the image data.

Each of the image pickup device 13A and the image pickup device 13B is connected to the image processing unit 104 by a parallel I/F bus. The image pickup device 13A and the image pickup device 13B are connected to the image pickup control unit 105 by a serial I/F bus (I2C bus or the like). The image processing unit 104 and the imaging control unit 105 are connected to the CPU 111 via the bus 110. Further, a ROM 112, SRAM 113, DRAM 114, operation unit 115, network I/F 116 and the like are also connected to the bus 110.

The image processing unit 104 takes in the image data of the hemispherical image output from the image pickup device 13A and the image pickup device 13B through a parallel I/F bus. Then, the image processing unit 104 synthesizes the image data of these hemispherical images to create the omnidirectional panoramic image 60.

The imaging control unit 105 sets commands and the like in the register group of the imaging elements 13A and 13B using the I2C bus with the imaging control unit 105 as a master device and the imaging elements 13A and 13B as slave devices. Necessary commands and the like are received from the CPU 111. Further, the imaging control unit 105 uses the I2C bus to take in status data and the like of the register group of the imaging device 13A and the imaging device 13B and send it to the CPU 111.

The imaging control unit 105 also functions as a synchronization control unit that cooperates with the CPU 111 to synchronize the output timings of the image data of the hemispherical images of the imaging elements 13A and 13B. In addition, in the present embodiment, the image capturing device 14 is not provided with a display unit, but a display unit may be provided.

The microphone 108 converts sound into sound (signal) data. The sound processing unit 109 takes in the sound data output from the microphone 108 through the I/F bus and performs a predetermined process on the sound data.

The CPU 111 controls the entire operation of the photographing device 14 and executes necessary processing. The ROM 112 stores various programs for the CPU 111. The SRAM 113 and the DRAM 114 are work memories and store programs executed by the CPU 111, data in the middle of processing, and the like. The DRAM 114 stores image data in the process of being processed by the image processing unit 104 and the processed spherical panoramic image 60.

The network I/F 116 is a general term for interface circuits (USB I/F, etc.) with external media such as SD cards and personal computers. Further, the network I/F 116 may be a network interface regardless of whether it is wireless or wired. The spherical panoramic image 60 stored in the DRAM 114 is recorded on an external medium via the network I/F 116 or transmitted to an external device such as the communication terminal 15 via the network I/F 116.

The communication unit 117 communicates with an external device such as the communication terminal 15 via the antenna 117a provided in the imaging device 14 by a short-range wireless technology such as Wi-Fi or NFC. The communication unit 117 may transmit the spherical panoramic image 60 to an external device of the communication terminal 15.

The electronic compass 118 detects shooting attitude information indicating the attitude (rotation angle (yaw, pitch, roll)) of the imaging device 14 from the magnetism of the earth. The electronic compass 118 may further include a GPS (Global Positioning System) function that indicates the current position of the imaging device 14.

Next, the hardware configurations of the server device 12 and the information processing terminal 18 will be described. FIG. 9 is an example of a hardware configuration diagram of the server device 12 and the information processing terminal 18.

The server device 12 and the information processing terminal 18 include a CPU 501, a ROM 502, a RAM 503, a HDD (Hard Disk Drive) 505, a media drive 507, a display 508, a network I/F 509, a keyboard 511, a mouse 512, and a CD-ROM drive 514. The configuration is such that they are connected via a bus 510. The media drive 507 reads or writes (stores) data with respect to the recording medium 506 such as a flash memory. The CD-ROM drive 514 reads or writes various data from or into a CD-ROM (Compact Disc Read Only Memory) 513 as an example of a removable recording medium. It should be noted that the server device 12 and the information processing terminal 18 may be configured to include a touch panel in which the display 508, the keyboard 511, and the mouse 512 are integrally configured.

Next, an outline of the flow of information processing performed by the information processing system 10 of the present embodiment will be described. FIG. 10 is an explanatory diagram showing an example of an outline of the flow of information processing performed by the information processing system 10.

In the present embodiment, the image capturing device 14 transmits the omnidirectional panoramic image 60 obtained by photographing and the related information 61 corresponding to the omnidirectional spherical panoramic image 60 via the communication terminal 15 and the wireless router 16. , To the server device 12 (steps S1 and S2).

In the present embodiment, the imaging device 14 will be described as being fixed at a predetermined position. Further, it is assumed that the image capturing device 14 performs continuous shooting and sequentially transmits each of the spherical panoramic images 60 obtained by the continuous shooting to the server device 12.

That is, in the present embodiment, the imaging device 14 will be described as sequentially transmitting a plurality of (several frames) spherical panoramic images 60 that are continuous in time series to the server device 12.

Further, the image capturing device 14 transmits the related information 61 corresponding to the spherical panoramic image 60 to the server device 12 together with the spherical panoramic image 60. That is, the imaging device 14 transmits the related information 61 corresponding to each of the spherical panoramic images 60 of each frame to the server device 12 together with the spherical panoramic image 60.

The related information 61 includes shooting date/time information indicating the shooting date/time of the corresponding omnidirectional panoramic image 60, and shooting device identification information (hereinafter, referred to as shooting device ID).

The image capturing device ID included in the related information 61 is identification information for identifying the image capturing device 14 that has captured the spherical panoramic image 60 corresponding to the related information 61.

Upon receiving the omnidirectional panoramic image 60 and the related information 61 from the imaging device 14, the server device 12 stores the omnidirectional panoramic image 60 and the related information 61. Then, the server device 12 performs a position specifying process (step S3). The position specifying process is a process of specifying the position of a person included in the spherical panoramic image 60 (details will be described later).

On the other hand, the information processing terminal 18 transmits an analysis result acquisition request to the server device 12 (step S4). The information processing terminal 18 receives an operation instruction from the user and transmits an analysis result acquisition request such as the position of a person in one or more omnidirectional panoramic images 60 to the server device 12.

Upon receiving the analysis result acquisition request, the server device 12 transmits the analysis result information including the position of the identified person to the information processing terminal 18 (step S5). The information processing terminal 18 displays the position and the like of the person shown in the received analysis result information. It should be noted that the display of the position of the person and the like shown in the analysis result information may be performed on the server device 12 side.

Next, the function of each device constituting the information processing system 10 of the present embodiment will be described in detail.

FIG. 11 is a schematic diagram showing a functional configuration of each of the server device 12, the imaging device 14, the communication terminal 15, and the information processing terminal 18 in the information processing system 10.

The imaging device 14 includes an imaging unit 14E, a storage unit 14F, a connection unit 14G, and a storage/readout unit 14I. The photographing unit 14E acquires image data of a hemispherical image by photographing and stores the image data in the storage unit 14F. The image capturing unit 14E reads out image data of a pair of hemispherical images at the same image capturing timing from the storage unit 14F and generates a spherical panoramic image 60. The storage/readout unit 14I also reads out the shooting date/time information of the generated spherical panorama image 60 and the shooting device ID of the shooting device 14 from the storage unit 14F as the related information 61. Then, the connection unit 14G transmits the omnidirectional panoramic image 60 generated by the imaging unit 14E and the related information 61 to the communication terminal 15.

The communication terminal 15 includes a connection unit 15A and a transmission/reception unit 15B. The connecting portion 15A is connected to the connecting portion 14G of the image capturing device 14. The connection unit 15A receives the omnidirectional panoramic image 60 and the related information 61 from the imaging device 14. The transmitter/receiver 15B communicates with the imaging device 14 via the connector 15A. Further, the transmission/reception unit 15B communicates with the server device 12 via the network 20. In the present embodiment, the transmitting/receiving unit 15B sends the spherical panoramic image 60 and related information 61 acquired from the imaging device 14 via the connection unit 15A to the server via the wireless router 16 (see FIG. 1) and the network 20. Send to device 12.

The server device 12 includes a control unit 12A, a storage unit 12B, a display unit 12F, and an input unit 12G. The storage unit 12B, the display unit 12F, the input unit 12G, and the control unit 12A are connected to each other so that data and signals can be exchanged.

The display unit 12F displays various images. The display unit 12F is, for example, an LCD or the like. The display unit 12F is realized by, for example, the display 508 (see FIG. 9). The input unit 12G receives an operation instruction from the user. The input unit 12G is, for example, a keyboard, a mouse, or the like. The input unit 12G is realized by, for example, a keyboard 511 or a mouse 512 (see FIG. 9).

The storage unit 12B stores various data. The storage unit 12B stores, for example, an image management DB (database) 12C and specific information 12H. The image management DB 12C stores a spherical panorama image 60, analysis information (details will be described later), and the like. The specific information 12H includes first specific information 12D and second specific information 12E.

FIG. 12 is a schematic diagram showing an example of the data structure of the first specific information 12D and the second specific information 12E.

FIG. 12A is a schematic diagram showing an example of the data structure of the first specific information 12D. The first specifying information 12D is information indicating a position specifying condition for specifying the position of the person according to the posture of the person. Specifically, the first identification information 12D associates the posture with the position identification condition.

The posture is the posture of the person. The posture is, for example, standing, sitting (chair sitting, squatting), supine, handstand, or the like. The standing position is a posture in which the user stands with the lower limb on the ground side so that the head, the trunk, and the four limbs are in a substantially straight line. The sitting position is a posture indicating a sitting state. There are several types of postures in sitting position. For example, sitting on a chair is a posture indicating a state of sitting on a chair. Crouching is a posture that indicates a crouched state. The supine position is a posture in which the back is on the ground side and the lower limbs are extended to lie down. The handstand is a posture in which the user stands with both hands and the head on the ground side and the lower limbs on the anti-ground side so that the head, the trunk, and the four limbs are in a substantially straight line.

The attitude registered in the first specific information 12D is not limited to these.

The position specifying condition corresponding to the posture is registered in the first specifying information 12D. The position specifying condition is a condition for specifying the position of a person having a corresponding posture. The position specifying condition registered in the first specifying information 12D is a condition for specifying a position that does not cause a sense of discomfort as the position of the person in the spherical panorama image 60 when the person in the corresponding posture is visually recognized. Is preferably shown.

In the example shown in FIG. 12A, "center position between both toes of a person" is registered as the position specifying condition corresponding to the posture "standing" in the first specifying information 12D. Further, in the first identification information 12D, "center position between both toes of a person" is registered as a position identification condition corresponding to each of the postures "sitting position" and "squatting".

Further, in the first identification information 12D, "center position of person area" is registered as a position identification condition corresponding to the posture "supine". In the first identification information 12D, "center position between both hands of person" is registered as a position identification condition corresponding to the posture "handstand".

As described above, in the first identification information 12D, the position identification condition for identifying the position of the person corresponding to each posture is registered in advance. The position specifying conditions registered in the first specifying information 12D include different ones depending on the posture. Therefore, the control unit 12A described later can specify a position corresponding to the posture of the person included in the spherical panoramic image 60 as the position of the person (details will be described later).

The posture and the corresponding position specifying condition in the first specifying information 12D can be changed by a user's operation instruction or the like (details will be described later).

Next, the second specific information 12E will be described. FIG. 12B is a schematic diagram showing an example of the data structure of the second specific information 12E. Position identification conditions corresponding to the posture of the person and the surrounding area are registered in the second identification information 12E. Specifically, the second identification information 12E associates the posture, the peripheral region, and the position identification condition with each other.

The posture of the second specific information 12E is the same as that of the first specific information 12D. The peripheral area is an area around the person area showing the person in the image (the spherical panoramic image 60 in the present embodiment). The peripheral area is an area within a predetermined range around the person area.

In the present embodiment, a case will be described in which the peripheral area indicates another object (hereinafter referred to as a peripheral object) existing around the person shown in the person area. In detail, in the present embodiment, the peripheral area is a peripheral object continuously arranged with respect to the person shown in the person area or a peripheral object arranged at a predetermined interval (discontinuously) with respect to the person. Indicates a thing. The peripheral objects are, for example, chairs, beds, futons, tables, shelves, and vehicles (automobiles, bicycles, carts, etc.).

The peripheral area is not limited to the case where the object around the person is shown, and may be the scene or environment around the person.

The position specifying condition registered in the second specifying information 12E is registered in association with the posture and peripheral objects. The position specifying condition in the second specifying information 12E is a condition for specifying the position of the person having the corresponding posture in which the corresponding peripheral object is arranged around. The position specifying condition registered in the second specifying information 12E is the position of the person in the spherical panorama image 60 when the person in the corresponding posture is visually recognized around the surrounding object. It is preferable to indicate a condition for specifying a position that does not cause discomfort.

In the example shown in FIG. 12B, the "center position between the toes of the person" is registered as the position specifying condition corresponding to the posture "standing" in the second specifying information 12E. In the second identification information 12E, "center position of tangent line between person area and chair area" is registered as the position identification condition corresponding to the posture "chair sitting" and the peripheral object "chair".

Further, in the second identification information 12E, "center position of person area" is registered as the position identification condition corresponding to the posture "supine" and the peripheral "bed, futon".

As described above, in the second identification information 12E, the position identification condition for identifying the position of the person according to the posture and the surrounding objects is registered in advance. Then, the position specifying conditions registered in the second specifying information 12E include different ones depending on the posture and peripheral objects. Therefore, the control unit 12A, which will be described later, can specify the position of the person included in the omnidirectional panorama image 60 according to the posture and surrounding objects as the position of the person (details will be described later).

It should be noted that the posture, peripheral objects, and corresponding position specifying conditions in the second specifying information 12E can be changed by an operation instruction from the user (details will be described later).

Next, the data structure of the image management DB 12C will be described. FIG. 13 is a schematic diagram showing an example of the data configuration of the image management DB 12C. The image management DB 12C is a database for managing captured image information and analysis information. The data structure of the image management DB 12C is not limited to the database and may be a table or the like.

The captured image information is information regarding the omnidirectional panoramic image 60 captured by each of the capturing devices 14. In the example illustrated in FIG. 13, the captured image information includes an image ID for identifying the spherical panoramic image 60 captured by the capturing device 14, a spherical panoramic image 60 identified by the image ID, and a spherical image. It includes a photographing device ID for identifying the photographing device 14 that photographed the spherical panoramic image 60, and photographing date and time information of the omnidirectional spherical panoramic image 60.

Every time the control unit 12A of the server device 12 receives the omnidirectional panoramic image 60 and the related information 61 from the imaging device 14, the received omnidirectional panoramic image 60, the imaging device ID and the imaging included in the related information 61. The date and time information is associated and registered in the image management DB 12C. At this time, the control unit 12A adds an image ID to the omnidirectional panoramic image 60 and registers it in the image management DB 12C. As a result, the captured image information in the image management DB 12C is sequentially updated.

The analysis information is information indicating the analysis result of the spherical panoramic image 60 by the control unit 12A. The analysis information is registered in the image management DB 12C by the processing described later by the control unit 12A.

The analysis information includes a person area ID, area information, a posture, peripheral objects, and a person's position.

The person area ID is identification information of each person area included in the corresponding spherical panoramic image 60. The person area is an area showing a person in the spherical panoramic image 60. The person area may be any area that includes a person in the spherical panoramic image 60. Specifically, the person area may be an area inside the outline of the person in the omnidirectional panorama image 60, or may be an area surrounding the person in a predetermined shape (for example, a rectangular shape). .. In the present embodiment, a case will be described as an example where the person area is an area that surrounds each person in the spherical panoramic image 60 individually (one by one) in a rectangular shape.

The area information is information indicating the position coordinates and the size of the person area identified by the corresponding person area ID in the corresponding omnidirectional panoramic image 60. In the example illustrated in FIG. 13, the person area has a position coordinate of one of four vertices (for example, an upper left apex of the rectangular person area) formed by a rectangle indicating the person area, and a first direction. (One direction in the two-dimensional plane of the spherical panoramic image 60, for example, the lateral direction) and a second direction orthogonal to the first direction (the two-dimensional plane of the spherical panoramic image 60, The length in the direction orthogonal to the direction 1 (for example, the vertical direction) can be expressed as follows.

The posture included in the analysis information in the image management DB 12C is the posture of the person included in the person area identified by the corresponding person area ID determined by the control unit 12A. The peripheral object (peripheral area) included in the analysis information in the image management DB 12C is the peripheral object of the person area specified by the control unit 12A.

The position of the person included in the analysis information in the image management DB 12C is the position in the corresponding omnidirectional panoramic image 60 of the person included in the person region identified by the corresponding person region ID specified by the control unit 12A. Is. The position of the person registered in the image management DB 12C may be information indicating the position in the spherical panorama image 60. In the present embodiment, the position of the person registered in the image management DB 12C is information indicating the position coordinate (x coordinate, y coordinate) of the person in the spherical panorama image 60. The x coordinate is the coordinate in the first direction (for example, the horizontal direction) on the two-dimensional plane of the spherical panoramic image 60. The y coordinate is the coordinate in the second direction (for example, the vertical direction) orthogonal to the first direction on the two-dimensional plane of the spherical panoramic image 60.

Returning to FIG. 11, the description will be continued. Next, the control unit 12A of the server device 12 will be described. The control unit 12A controls the server device 12. The control unit 12A is realized by the CPU 501, the ROM 502, the RAM 503, and the like (see FIG. 9). The control unit 12A may be realized by a circuit or the like.

Programs for executing various processes executed by the server device 12 according to the present embodiment are provided by being incorporated in the ROM 502 or the like in advance.

The program for executing various processes executed by the server device 12 of the present embodiment is a file in a format installable in the server device 12 or an executable format, which is a CD-ROM, a flexible disk (FD), It may be configured to be provided by being recorded in a computer-readable recording medium such as a CD-R or a DVD (Digital Versatile Disk).

Further, the program executed by the server device 12 of the present embodiment may be stored in a computer connected to a network such as the Internet and may be provided by being downloaded via the network. Further, the program for executing each process in the server device 12 of the present embodiment may be provided or distributed via a network such as the Internet.

The control unit 12A includes a transmission/reception unit 12K, a storage control unit 12P, a person area detection unit 12M, a posture determination unit 12J, a position identification unit 12Q, a change unit 12S, a periphery identification unit 12R, and an overlap determination unit 12T. A moving body discrimination unit 12X, a display control unit 12V, a reception unit 12W, a horizontal correction unit 12Y, and a distortion correction unit 12Z.

Transmission/reception unit 12K, storage control unit 12P, person area detection unit 12M, posture determination unit 12J, position identification unit 12Q, change unit 12S, periphery identification unit 12R, superposition determination unit 12T, moving body determination unit 12X, display control unit 12V, Part or all of the reception unit 12W, the horizontal correction unit 12Y, and the distortion correction unit 12Z may be realized by causing a processing device such as the CPU 501 (see FIG. 9) to execute a program (that is, software). .. Further, it may be realized by hardware such as an IC (Integrated Circuit) or may be realized in combination.

The transmission/reception unit 12K transmits/receives data to/from each of the imaging device 14 and the information processing terminal 18 via the network 20.

For example, the transmission/reception unit 12K receives the omnidirectional panoramic image 60 and the related information 61 from the imaging device 14 via the communication terminal 15, the wireless router 16, and the network 20. The transmission/reception unit 12K outputs the received spherical panoramic image 60 and related information 61 to the storage control unit 12P.

Further, the transmission/reception unit 12K receives the analysis result acquisition request from the information processing terminal 18. The transmitter/receiver 12K outputs the received analysis result acquisition request to the storage controller 12P. The transmitter/receiver 12K also transmits the analysis result information to the information processing terminal 18.

The storage control unit 12P stores various data in the storage unit 12B and reads various data from the storage unit 12B.

When the storage control unit 12P receives the omnidirectional panoramic image 60 and the related information 61 from the imaging device 14 via the transmitting/receiving unit 12K, the storage control unit 12P stores the omnidirectional panoramic image 60 and the related information 61 in the storage unit 12B. Specifically, the storage control unit 12P associates the received spherical panoramic image 60 with the shooting date/time information and the shooting device ID included in the related information 61 and registers them in the image management DB 12C (see FIG. 13). ), and the storage control unit 12P assigns an image ID to the omnidirectional panoramic image 60 and registers it in association with the image management DB 12C.

The storage control unit 12P also updates the image management DB 12C according to the processing of each functional unit included in the control unit 12A (details will be described later).

The horizontal correction unit 12Y performs correction so that the horizontal direction in the spherical panoramic image 60 received from the image capturing device 14 and the horizontal direction in the capturing environment match. As described above, when the omnidirectional panoramic image 60 received from the imaging device 14 is the omnidirectional panoramic image 60 represented by the Mercator projection method, the direction along the horizontal direction in the photographic environment is the long axis direction. There are cases where the spherical panoramic image 60 is not obtained. Here, the spherical panoramic image 60 represented by the Mercator projection is formed by projecting two 180° hemispherical images onto the inner wall of a virtual cylindrical object. Therefore, the long axis direction is a direction along the circumferential direction of the cylindrical object.

Therefore, the horizontal correction unit 12Y performs correction so that the long-axis direction in the omnidirectional panoramic image 60 received from the image capturing device 14 and the horizontal direction in the image capturing environment match. The horizontal correction unit 12Y may correct the horizontal direction using a known method.

The distortion correction unit 12Z corrects the distortion of the spherical panoramic image 60. As described above, the spherical panoramic image 60 represented by the Mercator projection is formed by projecting two 180° hemispherical images onto the inner wall of the virtual cylindrical object. For this reason, the distortion becomes larger as it gets closer to both ends of the spherical panorama image 60 represented by the Mercator projection in the long axis direction (direction matching the circumferential direction of the virtual cylindrical object). Therefore, the distortion correction unit 12Z corrects this distortion. The distortion correction unit 12Z may correct the distortion of the omnidirectional panoramic image 60 using a known method.

Then, the storage control unit 12P registers, in the image management DB 12C, the omnidirectional panoramic image 60 received from the image capturing device 14, which is the omnidirectional panoramic image 60 corrected by the horizontal correction unit 12Y and the distortion correction unit 12Z. do it.

As a result, other functional units that use the spherical panoramic image 60 stored in the storage unit 12B can use the spherical panoramic image 60 corrected by the horizontal correction unit 12Y and the distortion correction unit 12Z. ..

Note that the server device 12 may receive a spherical spherical panoramic image 60 (see FIG. 5) in which a hemispherical image is attached to the inner wall of a three-dimensional solid sphere from the image capturing device 14. In this case, the horizontal correction unit 12Y and the distortion correction unit 12Z do not have to perform the correction.

Next, the person area detection unit 12M, the posture determination unit 12J, the position identification unit 12Q, the periphery identification unit 12R, the superposition determination unit 12T, and the moving body determination unit 12X will be described. In the person area detecting unit 12M, the posture determining unit 12J, the position specifying unit 12Q, the peripheral specifying unit 12R, the superposition determining unit 12T, and the moving body determining unit 12X, the transmitting/receiving unit 12K takes a new spherical panoramic image 60. The process is performed each time it is received from the device 14.

As described above, the omnidirectional panoramic image 60 may be corrected by the horizontal correction unit 12Y and the distortion correction unit 12Z. In this case, the person area detection unit 12M, the posture determination unit 12J, the position identification unit 12Q, the peripheral identification unit 12R, the superposition determination unit 12T, and the moving body determination unit 12X are transmitted/received by the transmission/reception unit 12K. Every time it is received, the processing is executed on the spherical panoramic image 60 corrected by the horizontal correction unit 12Y and the distortion correction unit 12Z.

The person area detection unit 12M is an example of a detection unit. The person area detection unit 12M detects a person area included in the omnidirectional panoramic image 60. A known method may be used to detect the person area.

FIG. 14 is an explanatory diagram of detection of the person area 62.

For example, assume that the person area detection unit 12M detects the person area 62 included in the omnidirectional panoramic image 60A. The person area detection unit 12M detects an area including the person M included in the omnidirectional panoramic image 60A as the person area 62. Specifically, the person area detection unit 12M specifies the position coordinates and the size of the rectangular area showing the person M in the omnidirectional panoramic image 60A. In the example shown in FIG. 14, the case where the coordinates of the upper left vertex of the person area 62 (for example, the vertex (x ₁ , y ₁ )) is used as the position coordinates of the person area 62 is shown. In addition, in the example illustrated in FIG. 14, the case where the length (w) in the x-axis direction and the length (h) in the y-axis direction of the person area 62 are used as the size of the person area 62 is shown. The x-axis direction coincides with the long-axis direction described above.

As a result, the person area detection unit 12M detects the person area 62 in the spherical panoramic image 60A. When the omnidirectional panoramic image 60A includes a plurality of person areas 62, the person area detecting unit 12M detects the plurality of person areas 62.

Then, the person area detection unit 12M associates with the image ID of the celestial sphere panorama image 60 (the celestial sphere panorama image 60A in FIG. 14) to be processed, and the area information (position The coordinates and size) and the person area ID are registered in the image management DB 12C via the storage controller 12P. It should be noted that the person area detecting unit 12M may add information that can identify each of the person areas 62 as a person area ID and register the information in the image management DB 12C.

Returning to FIG. 11, the description will be continued. The posture determination unit 12J determines the posture of the person M shown in the person area 62 detected by the person area detection unit 12M. The posture determination unit 12J may determine the posture of the person M by using known image processing. In the present embodiment, the posture determination unit 12J determines any posture registered in the specific information 12H (see FIG. 12) as the posture of the person M shown in the person area 62.

Then, the specific information 12H registers the determined posture in the image management DB 12C via the storage control unit 12P in association with the person area ID of the person area 62 (see FIG. 13).

Note that the posture determination unit 12J may not be able to determine the posture of the person M shown in the person area 62. In such a case, the posture determination unit 12J may leave the column of the posture corresponding to the person area ID of the person area 62 in the image management DB 12C blank.

The position specifying unit 12Q specifies the position of the person M in the omnidirectional panorama image 60 based on the posture of the person M shown in the person area 62, which is determined by the posture determining unit 12J.

Specifically, the position specifying unit 12Q reads the position specifying condition corresponding to the posture judged by the posture judging unit 12J in the first specifying information 12D (see FIG. 12A). Then, the position specifying unit 12Q specifies the position specified by the position specifying condition in the spherical panoramic image 60 as the position of the person M.

Specifically, the position specifying unit 12Q performs the following process on each of the person areas 62 specified by the person area detecting unit 12M in the omnidirectional panorama image 60 to be processed. First, the position specifying unit 12Q reads, from the first specifying information 12D, the position specifying condition corresponding to the posture determined by the posture determining unit 12J for the person M shown in the person area 62.

Then, the position specifying unit 12Q determines the position (specifically, the position coordinate) specified by the position specifying condition read from the first specifying information 12D in the person area 62 of the spherical panoramic image 60 to be processed. It is specified as the position of the person M shown in the person area 62 in the spherical panoramic image 60.

For example, it is assumed that the posture determination unit 12J has determined the posture "standing" as the posture of the person M included in the human region 62. In this case, the position specifying unit 12Q reads the position specifying condition “center position between both toes” corresponding to the posture “standing” in the first specifying information 12D (see FIG. 12A). Then, the position specifying unit 12Q specifies the position coordinate corresponding to the center position between the toes of the person M shown in the person area 62 in the omnidirectional panorama image 60 as the position of the person M.

Here, as described above, the posture determination unit 12J may not be able to determine the posture of the person M shown in the person area 62. In this case, the position specifying unit 12Q specifies a predetermined position in the person area 62 as the position of the person M.

The predetermined position may be set in advance. For example, the center position of the person area 62 is preset as the predetermined position. When the posture of the person M shown in the person area 62 is not determined, the position identifying unit 12Q determines the position coordinates of the center position of the person area 62 in the omnidirectional panorama image 60 as the position of the person M. Should be specified as

Note that the position specifying unit 12Q may specify the position of the person M based on the posture of the person M and the peripheral region of the person M (peripheral object in the present embodiment). The peripheral object (peripheral area) of the person M is specified by the peripheral specifying unit 12R.

The peripheral specifying unit 12R specifies a peripheral area around the human area 62 in the spherical panoramic image 60. In the present embodiment, the peripheral specifying unit 12R specifies peripheral objects of the person M shown in the person area 62. A known method may be used to identify the peripheral object. For example, the peripheral specifying unit 12R specifies a chair, a bed, and the like as peripheral objects of the person M. Then, the position specifying unit 12Q may specify the position of the person M based on the posture of the person M and the peripheral objects of the person M.

In this case, the peripheral specifying unit 12R may specify the position of the person M using the second specifying information 12E (see FIG. 12B).

Specifically, the position identifying unit 12Q performs the following process on each of the person regions 62 in the omnidirectional panoramic image 60 to be processed. First, the position specifying unit 12Q determines the position specifying condition corresponding to the posture determined by the posture determining unit 12J and the peripheral object specified by the peripheral specifying unit 12R for the person M shown in the person region 62. The second specific information 12E is read (see FIG. 12B).

Therefore, in this case, the position specifying unit 12Q specifies the position in the human area 62 of the omnidirectional panorama image 60 to be processed by the position specifying condition corresponding to the posture of the person M shown in the person area 62 and the peripheral object. Position is specified as the position of the person M in the spherical panoramic image 60.

For example, it is assumed that the posture determination unit 12J determines the posture "chair sitting" as the posture of the person M included in the human region 62 detected by the human region detection unit 12M. In addition, it is assumed that the peripheral specifying unit 12R specifies a "chair" as a peripheral object of the person M.

In this case, the position specifying unit 12Q determines, based on the second specifying information 12E (FIG. 12(B)), the position specifying condition “person area and chair area” corresponding to the posture “chair sitting” and the peripheral object “chair”. Then, the position specifying unit 12Q corresponds to the center position of the tangent line between the person area 62 and the chair area indicating the chair on which the person M is sitting in the omnidirectional panorama image 60. The position coordinates to be specified are specified as the position of the person M.

FIG. 15 is an explanatory diagram illustrating an example of specifying the position of the person M.

For example, assume that the spherical panoramic image 60 to be processed is the spherical panoramic image 60B shown in FIG. In this case, the person area detection unit 12M detects each person area 62 (person area 62A to person area 62E) of the person M (person M1 to person M5). Then, the posture determination unit 12J determines the posture of each person M included in each of these person regions 62. In the example illustrated in FIG. 15, for example, the posture determination unit 12J determines the postures of the person M1 to the person M5 to be “standing position”, “sitting position”, “squatting”, “supine position”, and “handstand”, Discriminate each of.

The peripheral specifying unit 12R specifies the peripheral object D for each of the person areas 62 (person areas 62A to 62E). In the example illustrated in FIG. 15, the peripheral specifying unit 12R does not specify the peripheral object D for each of the person area 62A, the person area 62C, and the person area 62D. This is because the peripheral object D cannot be specified. On the other hand, the perimeter specifying unit 12R specifies the peripheral D2 indicating "chair" for the person area 62B and the peripheral D5 indicating "bed" for the person area 62E.

Then, the position specifying unit 12Q specifies the position of the person M in the omnidirectional panorama image 60B based on the posture of the person M and the peripheral object D shown in the person area 62. Note that the position specifying unit 12Q may specify the position of the person M based on only the posture of the person M.

Specifically, for the person area 62A, the position specifying unit 12Q, based on the second specifying information 12E, the position specifying condition “corresponding to the posture “standing” of the person M1 in the omnidirectional panoramic image 60B. The position P1 specified by the "central position between both toes" is specified as the position P1 of the person M1. For the person area 62B, the position specifying unit 12Q specifies the position corresponding to the posture "chair sitting" of the person M2 and the peripheral D2 "chair" in the spherical panorama image 60B based on the second specifying information 12E. The position P2 specified by the condition “center position of the tangent line between the person area and the chair area” is specified as the position P2 of the person M2.

Similarly, for the person area 62C, the position specifying unit 12Q, based on the second specifying information 12E, the position specifying condition “between the toes” corresponding to the posture “squat” of the person M3 in the omnidirectional panorama image 60B. The position P3 specified by the "central position of" is specified as the position P3 of the person M3. Similarly, for the person region 62D, the position specifying unit 12Q, based on the second specifying information 12E, the position specifying condition “between the hands” corresponding to the posture “handstand” of the person M4 in the spherical panoramic image 60B. The position P4 specified by the "central position" of the person M4 is specified as the position P4 of the person M4.

Further, the position identifying unit 12Q corresponds to the posture “supine position” and the peripheral D5 “bed” of the person M5 in the omnidirectional panoramic image 60B for the person area 62E based on the second identification information 12E. The position P5 specified by the position specifying condition "center position of tangent line between person area and chair area" is specified as the position P5 of the person M5.

Returning to FIG. 11, the description will be continued. Then, the position specifying unit 12Q registers the specified position of the person M in the image management DB 12C via the storage control unit 12P. Specifically, the position specifying unit 12Q registers the position coordinates indicating the position of the specified person M in the image management DB 12C in association with the image ID and the person area ID of the corresponding omnidirectional panoramic image 60.

Therefore, in the image management DB 12C, position coordinates indicating the position of the person M specified by the position specifying unit 12Q are registered for each of the person regions 62 included in the omnidirectional panoramic image 60.

The position identifying unit 12Q further determines whether another object is superimposed or non-superimposed on the person M shown in the person area 62, or whether the other superimposed object is a moving body or non-moving object. The position of the person M may be specified depending on whether the body is a body.

The other object superimposed on the person M is an object photographed in a state in which it is superimposed on the person M in the spherical panoramic image 60. That is, the other object superimposed on the person M is the peripheral object D of the person M and the peripheral object D superimposed on the person M.

In other words, the other object superimposed on the person M is the peripheral object D photographed in a state of being superimposed on the person M who is the target whose position P is specified. That is, the other object is arranged at a position closer to the photographing device 14 than the person M who is the target for specifying the position P in the photographing angle of view from the photographing position of the photographing device 14, and a part of the person M is defined. It is the peripheral object D arranged at the position to be shielded. The other object (peripheral object D) is, for example, another person, a vehicle, a shelf, a bed, or the like.

In this case, the position specifying unit 12Q may specify the position of the person M by further using the determination result by the superposition determination unit 12T and the determination result by the moving body determination unit 12X.

The superposition determining unit 12T determines whether the peripheral object D (that is, another object) is superposed or non-superposed on the person M shown in the person area 62 in the omnidirectional panoramic image 60. The superposition determining unit 12T may use a known image processing technique to determine whether the peripheral object D is superposed on the person M or is not superposed.

The moving body discriminating unit 12X discriminates whether the peripheral object D superimposed on the person M is a moving body or a non-moving body.

The moving body refers to an object that has not been positioned on the person M continuously for a predetermined time or longer. In other words, the moving body is a peripheral object D other than the peripheral object D that is superposed on the person M and that exists on the person M continuously for a predetermined time or longer. For example, the moving body is a moving vehicle, a moving creature, or the like. The non-moving object is a peripheral object D that is superposed on the person M and that is kept superposed on the person M for a predetermined time or longer.

For example, the moving object discrimination unit 12X is photographed before (before) the spherical panoramic image 60 to be processed and by the photographing device 14 that photographed the spherical panoramic image 60. The peripheral object D superimposed on the person M is determined to be a moving object or a non-moving object by comparing with the spherical panoramic image 60.

For example, the moving object determination unit 12X determines that the position of the peripheral object D, which is included in the omnidirectional panoramic image 60 to be processed and is superimposed on the person M, is the position in the omnidirectional panoramic image 60 that was previously captured. It is determined whether or not it is located at the same position as. As a result, the moving body discriminating unit 12X discriminates the peripheral object D as a non-moving body when it is located at the same position. In addition, the moving body discriminating unit 12X discriminates the peripheral object D as a moving body when it is located at different positions. In addition, the moving body determination unit 12X may use another method to determine whether the peripheral object D is a moving body or a non-moving body.

Then, when it is determined that the peripheral object D is superposed on the person M shown in the person area 62 and the peripheral object D is a non-moving object, the position specifying unit 12Q determines the person area. The predetermined position of 62 is specified as the position of the person M. The predetermined position may be set in advance. For example, the position of the person M shown in the person area 62 in the exposed area exposed from the peripheral object D may be predetermined as the predetermined position.

Further, when it is determined that the peripheral object D is superimposed on the person M shown in the person area 62 and the peripheral object D is a moving body, the position specifying unit 12Q determines the person area 62. The position of the person M in the spherical panoramic image 60 is specified based on the posture of the person M shown in FIG. The method of specifying the position based on the posture is the same as above.

Further, when it is determined that the peripheral object D is not superposed on the person M shown in the person area 62, the position specifying unit 12Q determines the person M in the omnidirectional panoramic image 60 based on the posture of the person M. Specify the position of. The method of specifying the position based on the posture is the same as above.

This will be described in detail with reference to the drawings. 16 and 17 are explanatory diagrams showing an example in which the position of the person M is specified according to the peripheral object D superimposed on the person M shown in the person area 62.

For example, assume that the spherical panoramic image 60D shown in FIG. 16B is the spherical panoramic image 60 to be processed. In this case, the person area detection unit 12M detects, for example, each person area 62 (person area 62F, person area 62G) of the person M (person M6, person M7).

Then, the superposition determining unit 12T determines whether the peripheral object D is superposed or non-superposed on the person M (person M6, person M7) shown in the detected person area 62 (person area 62F, person area 62G). To determine. In the case of the omnidirectional panoramic image 60D shown in FIG. 16B, the superposition determining unit 12T determines that the peripheral object D is not superposed on the person M6. Further, the superimposition determining unit 12T determines that the peripheral object D7 indicating the shelf is superposed on the person M7.

Then, the moving body discriminating unit 12X discriminates whether the peripheral object D7 superposed on the person M7 discriminated by the superimposing discriminating unit 12T is a moving body or a non-moving body. For example, the moving object discriminating unit 12X detects the omnidirectional panoramic image 60C (FIG. 16(A)) taken before the omnidirectional panoramic image 60D by the imaging device 14 that has taken the omnidirectional panoramic image 60D. , From the image management DB 12C of the storage unit 12B. Then, the moving object discriminating unit 12X discriminates whether or not the peripheral object D7 exists at the position corresponding to the peripheral object D7 of the spherical panoramic image 60D in the spherical panoramic image 60C. Then, when the peripheral object D7 exists at a position corresponding to the peripheral object D7 of the spherical panoramic image 60D in the spherical panoramic image 60C, the moving object determination unit 12X determines the peripheral object D7 as a non-moving object. It is determined that

Then, for the person M7 on which the non-moving object D7 is superimposed in the omnidirectional panoramic image 60D, the position identifying unit 12Q sets a predetermined position in the person area 62G of the person M7 as the position of the person M7. Specified as P7. Specifically, in this case, for the person M7 on which the peripheral object D7 such as a shelf of a non-moving object is superimposed, in this case, the position specifying unit 12Q sets a predetermined position in the exposed region exposed from the peripheral object D7 of the person M7 as the person. It is specified as the position P7 of M7. Therefore, in this case, the position specifying unit 12Q does not specify the position, such as the foot of the person M7, which is not directly observed from the spherical panoramic image 60D, as the position P of the person M7. Then, the position specifying unit 12Q specifies the position of the person M7 within the area directly observed from the spherical panoramic image 60D as the position P7 of the person M7.

On the other hand, the peripheral object D is not superimposed on the person M6 in the spherical panoramic image 60D. Therefore, the position specifying unit 12Q specifies the position P6 of the person M6 based on the posture of the person M6. The method of identifying the position P based on the posture is the same as above.

Next, description will be made with reference to FIG. For example, assume that the spherical panoramic image 60F shown in FIG. 17B is the spherical panoramic image 60 to be processed. Then, it is assumed that the person area detection unit 12M has detected each person area 62 (person area 62H, person area 62I) of the person M (person M8, person M9).

In this case, the superimposition determining unit 12T determines whether or not the peripheral object D is superposed on the person M (person M8, person M9) shown in the detected person area 62 (person area 62H, person area 62I). Determine whether. In the case of the omnidirectional panoramic image 60F shown in FIG. 17B, the superposition determining unit 12T determines that the peripheral object D is not superposed on the person M8. In addition, the superimposition determining unit 12T determines that the peripheral object D8 indicating the person M8 is superimposed on the person M9.

Then, the moving body discriminating unit 12X discriminates whether the peripheral object D8 (person M8) superposed on the person M9 discriminated by the superposition discriminating unit 12T is a moving body or a non-moving body.

For example, the moving object discriminating unit 12X detects the omnidirectional panoramic image 60E (FIG. 17(A)) taken before the omnidirectional panoramic image 60F by the imaging device 14 that has taken the omnidirectional panoramic image 60F. , From the image management DB 12C of the storage unit 12B. Then, the moving body discriminating unit 12X discriminates whether or not the peripheral object D8 exists at a position in the spherical panoramic image 60E corresponding to the peripheral object D8 of the spherical panoramic image 60F. Then, when the peripheral object D8 does not exist at the position corresponding to the peripheral object D8 of the spherical panoramic image 60F in the spherical panoramic image 60E, the moving object discriminating unit 12X recognizes the peripheral object D8 as a moving object. Determine that there is. In the case shown in FIG. 17, the moving object discriminating unit 12X does not have the peripheral object D8 (person M8) in the omnidirectional panoramic image 60E. Therefore, the moving body determination unit 12X determines that the peripheral object D8 (person M8) is a moving body.

Then, for the person M9 on which the peripheral object D8, which is a moving body, in the omnidirectional panoramic image 60F is superimposed, the position specifying unit 12Q determines the person M9 in the omnidirectional panoramic image 60F based on the posture of the person M9. Position P9 is specified. The method of identifying the position P based on the posture is the same as above.

On the other hand, the peripheral object D is not superimposed on the person M8 in the spherical panoramic image 60F. Therefore, the position specifying unit 12Q specifies the position P8 of the person M8 based on the posture of the person M8. The method of identifying the position P based on the posture is the same as above.

Returning to FIG. 11, the description will be continued. The reception unit 12W receives an input from the input unit 12G by the user. In the present embodiment, reception unit 12W receives an instruction to change specific information 12H from input unit 12G. The accepting unit 12W may accept an instruction to change the specific information 12H from the information processing terminal 18 via the transmitting/receiving unit 12K.

For example, the display control unit 12V displays a setting screen for accepting the change of the specific information 12H on the display unit 12F. FIG. 18 is a schematic diagram showing an example of the setting screen 70. The setting screen 70 includes setting fields for setting each of the posture, the peripheral objects, and the position of the person. The user operates the input unit 12G while visually checking the setting screen 70, thereby inputting the position specifying condition corresponding to the desired posture and the peripheral object D.

The accepting unit 12W accepts the posture, the peripheral objects, and the position specifying condition input by the user as an instruction to change the specifying information 12H. The changing unit 12S changes the specific information 12H according to the change instruction. Specifically, the changing unit 12S registers the posture, the peripheral object, and the position specifying condition indicated in the change instruction in association with each other in the specifying information 12H in the storage unit 12B. As a result, the specific information 12H in the storage unit 12B is updated.

Returning to FIG. 11, the description will be continued. By performing the above processing by the control unit 12A, every time the spherical panoramic image 60 is received from the image capturing device 14, the spherical panoramic image 60 is registered in the image management DB 12C, and the spherical panoramic image 60 is registered. Analysis information (see FIG. 13) corresponding to the panoramic image 60 is registered.

Next, the functional configuration of the information processing terminal 18 will be described. The information processing terminal 18 includes a transmission/reception unit 18A, a reception unit 18C, a display control unit 18D, a storage/readout unit 18B, a storage unit 18E, an input unit 18F, a display unit 18G, an acquisition unit 18H, and a display. And an image generation unit 18I.

The transmission/reception unit 18A, the storage/readout unit 18B, the reception unit 18C, the display control unit 18D, the storage unit 18E, the acquisition unit 18H, and the display image generation unit 18I are connected so that data and signals can be exchanged.

The reception unit 18C is connected to the input unit 18F. The reception unit 18C receives an operation instruction of the input unit 18F by the user. The input unit 18F corresponds to the keyboard 511 and the mouse 512 (see FIG. 9). The display control unit 18D displays various images on the display unit 18G. The display unit 18G corresponds to the display 508 (see FIG. 9). The transmission/reception unit 18A communicates with the server device 12 via the network 20. The storage unit 18E stores various data. For example, the storage unit 18E stores the user ID of the user who operates the information processing terminal 18. The memory/reader 18B stores various data in the memory 18E and reads various data from the memory 18E.

The user inputs the analysis result acquisition request by operating the input unit 18F.

The analysis result acquisition request includes the image ID, the analysis result of the position of the person M in the omnidirectional panorama image 60 identified by the image ID, and the image ID of the omnidirectional panorama image 60 to be displayed (hereinafter referred to as representative image ID In some cases) and a request signal indicating a request to acquire the analysis result.

For example, the user inputs the analysis result acquisition request by operating the input unit 18F while referring to the display unit 18G.

When receiving the analysis result acquisition request from the input unit 18F, the reception unit 18C transmits the analysis result acquisition request to the server device 12 via the transmission/reception unit 18A. Upon receiving the analysis result acquisition request from the information processing terminal 18, the transmission/reception unit 12K of the server device 12 outputs it to the storage control unit 12P. The storage control unit 12P stores the omnidirectional panoramic image 60 corresponding to the image ID included in the received analysis result acquisition request, the shooting date/time information corresponding to the image ID, and the analysis information corresponding to the image ID. The image management DB 12C is read as analysis result information (see FIG. 13). Then, the transmission/reception unit 12K transmits the analysis result information to the information processing terminal 18, which is the transmission source of the analysis result acquisition request.

The transmission/reception unit 18A of the information processing terminal 18 receives the analysis result information from the server device 12 as a response to the analysis result acquisition request transmitted to the server device 12. The acquisition unit 18H acquires the analysis result information from the transmission/reception unit 18A. That is, the acquisition unit 18H indicates a position P indicating the image ID, the omnidirectional panoramic image 60 corresponding to the image ID, and the position M of the person M specified for each person region 62 included in the omnidirectional panoramic image 60. Get the coordinates and.

The display image generation unit 18I generates a display image. Specifically, the display image generation unit 18I reads the spherical panoramic image 60 included in the analysis result information. Then, a display image in which a drawing point indicating the person M is drawn at the position coordinates indicating the position P of the person M in the omnidirectional panoramic image 60 is generated.

FIG. 19 is a schematic diagram showing an example of the display image 72 (display image 72A). The display image 72A is a schematic diagram showing a state in which the spherical panoramic image 60B shown in FIG. 15 is displayed on the display unit 18G. As described with reference to FIG. 15, the position identifying unit 12Q of the server device 12 determines the person M1 to the person M5 included in the omnidirectional panorama image 60B according to the postures and peripheral objects D of the person M1 to the person M5. Each position P (position P1 to position P5) is specified.

Therefore, the display image generation unit 18I causes the drawing point B (drawing point B indicating the person M at the position indicated by the position coordinates of the person M corresponding to the person region ID included in the analysis information in the omnidirectional panoramic image 60B. B1 to drawing point B5) are drawn. In this way, the display image generating unit 18I draws the drawing points at the positions P (positions P1 to P5) based on the respective postures of the person M (people M1 to M5) included in the omnidirectional panoramic image 60B. A display image 72A in which each of B (drawing point B1 to drawing point B5) is drawn is generated.

Then, the display control unit 18D displays the display image 72 on the display unit 18G.

Here, the position coordinate indicating the position P of the person M used when the display control unit 18D created the display image 72 is the position P specified by the position specifying unit 12Q based on the posture of the person M. Therefore, the display control unit 18D can display the display image 72 that accurately represents the position P of the person M included in the spherical panoramic image 60.

The display image generation unit 18I may specify one of the plurality of spherical panoramic images 60 included in the analysis result information received from the server device 12 as the representative image to be displayed. The user may specify the representative image to be displayed by operating the input unit 18F. Then, the display image generation unit 18I may draw the drawing point B on the omnidirectional panoramic image 60 designated as the representative image, which is received from the input unit 18F via the reception unit 18C.

In this case, the display image generation unit 18I determines the position P of the person M corresponding to each of the omnidirectional panoramic image 60 and the other omnidirectional panoramic image 60 in the omnidirectional panoramic image 60 designated as the representative image. The display image 72 may be generated by drawing the drawing point B on each of these.

Accordingly, the display image generation unit 18I can generate the display image 72 in which the drawing point B indicating the heat map indicating the staying degree of the person M is drawn on the omnidirectional panoramic image 60 as the representative image. ..

Further, when drawing the drawing point B indicating the heat map, it is preferable that the display image generation unit 18I adjust the color of the drawing point B according to the density of the drawing points B. As a result, the drawing point B indicating the degree of stay of the person M can be drawn on the omnidirectional panoramic image 60 as the representative image. A known method may be used as the method for creating the heat map. However, in the present embodiment, the heat map is created using the position P based on the posture of the person M identified by the position identifying unit 12Q. Therefore, in the information processing system 10 of the present embodiment, it is possible to create a heat map that more accurately indicates the degree of staying of the person M.

The acquisition unit 18H and the display image generation unit 18I may be provided in the control unit 12A of the server device 12. That is, the display image 72 may be created on the control unit 12A side.

Next, a procedure of information processing executed by the server device 12 will be described. FIG. 20 is a flowchart showing an example of an information processing procedure executed by the control unit 12A of the server device 12.

First, the transmission/reception unit 12K determines whether or not the spherical panoramic image 60 and the related information 61 have been received from the imaging device 14 (step S100). If an affirmative decision is made in step S100 (step S100: Yes), the operation proceeds to step S102.

In step S102, the horizontal correction unit 12Y performs horizontal correction of the spherical panoramic image 60 received in step S100 (step S102). Next, the distortion correction unit 12Z corrects the distortion of the omnidirectional panoramic image 60 that has been horizontally corrected in step S102 (step S104).

Then, the storage control unit 12P registers the spherical panorama image 60 received in step S100 and corrected in steps S102 and S104 in the image management DB 12C (step S106). At this time, the storage control unit 12P registers the shooting date and time information and the shooting device ID included in the related information 61 received in step S100 in the image management DB 12C in association with the spherical panorama image 60. In addition, the storage control unit 12P assigns an image ID to the omnidirectional panoramic image 60, and associates the image ID with the image ID and registers it in the image management DB 12C.

For this reason, by the processing of steps S100 to S106, the image management DB 12C (see FIG. 13) has the omnidirectional panoramic image 60 that has been received from the imaging device 14 and has been subjected to horizontal correction and distortion correction, and the imaging device. The ID, the image ID, and the shooting date/time information are registered in association with each other. In other words, every time the omnidirectional panoramic image 60 is received from the image capturing device 14, new captured image information is registered in the image management DB 12C.

Next, the person area detection unit 12M detects the person area 62 included in the omnidirectional panorama image 60 registered in the image management DB 12C in step S106 (step S108). Then, the person area detection unit 12M gives a person area ID to each of the detected person areas 62. Further, the person area detection unit 12M associates the person area ID and the area information (positional coordinates, size) of the person area 62 identified by the person area ID with the image ID of the omnidirectional panoramic image 60. And register it in the image management DB 12C (see FIG. 13).

Next, the control unit 12A repeats the processes of steps S110 to S132 for the omnidirectional panoramic image 60 received in step S100 for each person region 62 detected in step S108.

In detail, the periphery specifying unit 12R specifies the peripheral object D around the person area 62 (step S110). Next, the periphery specifying unit 12R determines whether or not the peripheral object D has been specified in step S110 (step S112). When a negative determination is made in step S112 (step S112: No), the process proceeds to step S118 described below. If an affirmative decision is made in step S112 (step S112: Yes), the operation proceeds to step S114.

In step S114, it is determined whether the peripheral object D identified in step S112 is superposed or non-superposed on the person M in the person area 62 to be processed (step S114). When it is determined that the peripheral object D is not superposed on the person M (step S114: No), the process proceeds to step S118.

On the other hand, when it is determined that the peripheral object D is superimposed on the person M (step S114: Yes), the process proceeds to step S116. In step S116, the moving body determination unit 12X determines whether or not the peripheral object D identified in step S112 is a moving body (step S116).

When it is determined that the peripheral object D superimposed on the person M is a moving body (step S116: Yes), the process proceeds to step S118.

In step S118, the posture determination unit 12J determines the posture of the person M (step S118). Next, the posture determination unit 12J determines whether or not the posture of the person M can be determined (step S120). When the posture of the person M can be determined (step S120: Yes), the process proceeds to step S122.

In step S122, it is determined whether or not the peripheral object D has been identified by the process of step S110 (step S122). When the peripheral object D can be specified (step S122: Yes), the process proceeds to step S124.

In step S124, the position specifying unit 12Q specifies the position of the person M from the posture of the person M and the peripheral object D (step S124). Then, the process proceeds to step S132.

On the other hand, when it is determined in step S122 that the peripheral object D cannot be specified (step S122: No), the process proceeds to step S126. In step S126, the position specifying unit 12Q specifies the position of the person M from the posture of the person M (step S126). Then, the process proceeds to step S132.

On the other hand, when it is determined in step S120 that the posture of the person M cannot be determined (step S120: No), the process proceeds to step S128. In step S128, the position specifying unit 12Q specifies a predetermined position in the person area 62 as the position of the person M (step S128). Then, the process proceeds to step S132.

Further, when it is determined in the above step S116 that the peripheral object D superimposed on the person M is a non-moving object (step S116: No), the process proceeds to step S130. In step S130, the position specifying unit 12Q specifies the predetermined position of the person area 62 as the position of the person M (step S130). Then, the process proceeds to step S132.

In step S132, the storage control unit 12P sets the position coordinates indicating the position of the person M identified by the processing in steps S124 to S130 to the image ID of the omnidirectional panorama image 60 to be processed and the person area 62 to be processed. It is registered in the image management DB 12C in association with the person area ID of (step S132).

Then, the control unit 12A ends the repetition when the processes of steps S110 to S132 are repeated for all of the person regions 62 detected in step S108 for the omnidirectional panoramic image 60 received in step S100. Then, this routine ends.

On the other hand, when a negative determination is made in step S100 (step S100: No), the process proceeds to step S134. In step S134, the reception unit 12W determines whether or not an instruction to change the specific information 12H has been received (step S134).

If an affirmative decision is made in step S134 (step S134: Yes), the operation proceeds to step S136. In step S136, the changing unit 12S updates the specific information 12H according to the change instruction received in step S134 (step S136). Then, this routine ends.

On the other hand, when a negative determination is made in step S134 (step S134: No), the process proceeds to step S138. In step S138, the transmission/reception unit 12K determines whether or not an analysis result acquisition request has been received from the information processing terminal 18 (step S138). If an affirmative decision is made in step S138 (step S138: Yes), the operation proceeds to step S140.

In step S140, the storage control unit 12P reads the analysis result information including the captured image information and the analysis information in the image management DB 12C (step S140), and proceeds to step S142. In step S142, the transmission/reception unit 12K transmits the analysis result information read in step S140 to the information processing terminal 18 that is the transmission source of the analysis result acquisition request in step S138 (step S142). Then, this routine ends. If a negative decision is made in step S138 (step S138: No), this routine is ended.

Next, an example of an information processing procedure executed by the information processing terminal 18 will be described. FIG. 21 is a flowchart showing an example of an information processing procedure executed by the information processing terminal 18.

First, the reception unit 18C of the information processing terminal 18 receives the analysis result acquisition request (step S200). Next, the transmission/reception unit 18A transmits the analysis result acquisition request accepted in step S200 to the server device 12 (step S202).

Next, the acquisition unit 18H acquires the analysis result information from the server device 12 via the transmission/reception unit 18A (step S204). Next, the display image generation unit 18I draws the drawing point B indicating the person M at the position of the person M indicated by the analysis information in the omnidirectional panoramic image 60 included in the analysis result information. It is generated (step S206). As described above, the display image generation unit 18I causes the drawing point B to be located at the position of the person M corresponding to each of the plurality of spherical panoramic images 60 on the spherical panoramic image 60 designated as the representative image. The display image 72 may be generated by drawing.

Next, the display control unit 18D displays the display image 72 generated in step S206 on the display unit 18G (step S208). Then, this routine ends.

As described above, the server device 12 (information processing device) of the present embodiment includes the person area detection unit 12M (detection unit), the posture determination unit 12J, and the position identification unit 12Q. The person area detection unit 12M detects the person area 62 included in the omnidirectional panoramic image 60 (image). The posture determination unit 12J determines the posture of the person M shown in the person area 62. The position specifying unit 12Q specifies the position of the person M in the omnidirectional panoramic image 60 based on the posture.

As described above, in the server device 12 of the present embodiment, the position of the person M is specified based on the posture of the person M shown in the person area 62 included in the omnidirectional panoramic image 60.

Therefore, when the same position (for example, the center) in the person area 62 in the omnidirectional panoramic image 60 is specified as the position of the person M shown in the person area 62, the omnidirectional panoramic image 60 occurs. It is possible to prevent an unnatural portion from being identified as the position of the person M when visually recognizing.

As described above, the target image processed by the server device 12 is not limited to the spherical panoramic image 60.

Therefore, the server device 12 according to the present embodiment can accurately specify the position P of the person M in the image.

The peripheral specifying unit 12R specifies a peripheral area (for example, the peripheral object D) around the human area 62 in the omnidirectional panoramic image 60. The position specifying unit 12Q specifies the position of the person M in the omnidirectional panoramic image 60 based on the posture and the peripheral area (for example, the peripheral object D).

Therefore, in addition to the above effects, the server device 12 can specify the position P of the person M in the image with higher accuracy.

Further, when the posture of the person M shown in the person area 62 is not determined, the position specifying unit 12Q specifies a predetermined position in the person area 62 as the position P of the person M.

The superposition determining unit 12T determines whether another object is superposed or non-superposed on the person M shown in the person area 62 in the omnidirectional panoramic image 60. The moving body determination unit 12X determines whether the other object is a moving body or a non-moving body. If it is determined that another object is superposed on the person M shown in the person area 62 and that the other object is a non-moving object, the position identifying unit 12Q Is specified as the position P of the person M. Further, when it is determined that another object is superposed on the person M shown in the person area 62 and the other object is a moving body, the position specifying unit 12Q determines that the person area 62 is a moving body. The position of the person M in the spherical panoramic image 60 is specified based on the posture of the person M shown in FIG. Further, when it is determined that another object is not superposed on the person M shown in the person area 62, the position identifying unit 12Q determines the position of the person M in the omnidirectional panorama image 60 based on the posture of the person M. Specify.

The display image generation unit 18I generates the display image 72 in which the drawing point B is drawn at the position P of the person M specified by the position specifying unit 12Q in the omnidirectional panorama image 60 to be displayed.

The position specifying unit 12Q specifies the position corresponding to the determined attitude in the first specifying information 12D (specific information 12H) in which the attitude and the position specifying condition for specifying the position P of the person M are associated with each other. The position P specified by the condition is specified as the position P of the person M.

In addition, the position specifying unit 12Q is determined in the second specifying information 12E (specific information 12H) in which the posture, the peripheral area, and the position specifying condition for specifying the position P of the person M are associated with each other. The position P specified by the position specifying condition corresponding to the posture and the specified peripheral region is specified as the position P of the person M.

The reception unit 12W receives an instruction to change the specific condition information. The changing unit 12S changes the specific information 12H according to the change instruction.

The image is a omnidirectional panoramic image 60 obtained by photographing in all directions.

The horizontal correction unit 12Y corrects the horizontal panoramic image 60 so that the horizontal direction matches the horizontal direction in the shooting environment. The distortion correction unit 12Z corrects the distortion of the spherical panoramic image 60. The posture determination unit 12J determines the posture of the person M included in the omnidirectional panoramic image 60 as the corrected image. The position specifying unit 12Q specifies the position of the person M in the spherical panoramic image 60.

The information processing system 10 includes a server device 12 (information processing device) and an information processing terminal 18. The information processing terminal 18 includes a display control unit 18D. The display control unit 18D displays the display image 72 showing the position of the person M on the display unit 18G.

(Second embodiment)
In the present embodiment, a position (first position) based on the posture of the person M and a position (second position) based on the peripheral region of the person M are specified for each person region 62. The case will be described.

FIG. 1 is a configuration diagram of an information processing system 10A according to the present embodiment. The information processing system 10A is similar to the information processing system 10 of the first embodiment except that the server apparatus 11 is provided instead of the server apparatus 12 and the information processing terminal 19 is provided instead of the information processing terminal 18. Is.

The hardware configurations of the server device 11 and the information processing terminal 19 are similar to those of the information processing system 10 and the information processing terminal 18 of the first embodiment.

Next, the function of each device constituting the information processing system 10A of the present embodiment will be described in detail.

FIG. 22 is a schematic diagram showing the functional configurations of the server device 11, the imaging device 14, the communication terminal 15, and the information processing terminal 19 in the information processing system 10A. The configurations of the image capturing device 14 and the communication terminal 15 are the same as those in the first embodiment.

The server device 11 includes a control unit 11A, a storage unit 11B, a display unit 12F, and an input unit 12G. The storage unit 11B, the display unit 12F, the input unit 12G, and the control unit 11A are connected to each other so that data and signals can be exchanged. The display unit 12F and the input unit 12G are the same as those in the first embodiment.

The storage unit 11B stores various data. The storage unit 11B stores, for example, an image management DB (database) 11C and specific information 12H. The storage unit 11B is similar to the storage unit 12B of the first embodiment except that the image management DB 11C is stored instead of the image management DB 12C.

The image management DB 11C stores the spherical panoramic image 60, analysis information, and the like. FIG. 23 is a schematic diagram showing an example of the data configuration of the image management DB 11C. The image management DB 11C is a database for managing captured image information and analysis information. The data structure of the image management DB 11C is not limited to the database and may be a table or the like. The captured image information is the same as the captured image information in the image management DB 12C of the first embodiment.

The analysis information in the image management DB 11C is information indicating the analysis result of the spherical panoramic image 60 by the control unit 11A. The analysis information in the present embodiment is registered in the image management DB 11C by the processing to be described later by the control unit 11A.

The analysis information according to the present embodiment includes a person area ID, area information, a posture, peripheral objects, a first position, and a second position. The person area ID, area information, posture, and peripheral objects are the same as those in the first embodiment.

The first position and the second position are positions in the corresponding omnidirectional panoramic image 60 of the person included in the person area identified by the corresponding person area ID specified by the control unit 11A. ..

The first position is the position P of the person M based on the posture of the person M. That is, the first position in the present embodiment is the same as the position P of the person M specified by the position specifying unit 12Q in the first embodiment.

The second position indicates the position of the person M, and is a position that is independent of changes in the peripheral area around the person area. Therefore, in other words, the first position is a position that may (or may) depend on changes in the peripheral region around the person region. On the other hand, the second position is a position that does not change even when the peripheral region of the person M changes with time.

These first position and second position are specified by the control unit 11A described later and registered in the image management DB 11C.

The position of the person (first position, second position) registered in the image management DB 11C can be represented by position coordinates (x coordinate, y coordinate) in the spherical panoramic image 60.

Returning to FIG. 22, the description will be continued. Next, the control unit 11A of the server device 11 will be described. The control unit 11A controls the server device 11. The control unit 11A is the same as the control unit 12A of the first embodiment except that the position specifying unit 12Q and the storage control unit 12P are replaced by a position specifying unit 11Q and a storage control unit 11P.

The position specifying unit 11Q specifies the first position and the second position as the position of the person M shown in the person area 62 detected by the person area detecting unit 12M. The position specifying unit 11Q specifies the first position by specifying the position of the person M in the same manner as the position specifying unit 12Q of the first embodiment.

Regarding the second position, the position specifying unit 11Q stores in advance a position specifying condition that is independent of the peripheral object D (referred to as an independent dependence condition). Then, the position specifying unit 11Q specifies the position specified by the non-dependent specifying condition for the person M shown in the person area 62 specified by the person area detecting unit 12M as the second position of the person M.

For example, the position specifying unit 11Q stores in advance the "center position between the two toes" as the independent specifying condition. Then, the position identifying unit 11Q identifies, as the second position of the person M, a position corresponding to the “center position between the two toes” of the person M shown in the person region 62 specified by the person region detecting unit 12M. ..

As a result, the position specifying unit 11Q generates analysis information including the specified first position and the specified second position for each person region 62 included in the omnidirectional panoramic image 60.

The storage control unit 11P is the same as the storage control unit 12P of the first embodiment except that various types of information are registered in the image management DB 11C instead of the image management DB 12C. However, the storage control unit 11P stores the analysis information including the first position and the second position specified by the position specifying unit 11Q in the image management DB 11C.

Next, the functional configuration of the information processing terminal 19 will be described. The information processing terminal 19 includes a transmission/reception unit 18A, a reception unit 18C, a display control unit 18D, a storage/readout unit 18B, a storage unit 18E, an input unit 18F, a display unit 18G, an acquisition unit 19H, and a display. And an image generation unit 19I.

The information processing terminal 19 is the same as the information processing terminal 19 of the first embodiment except that an acquisition unit 19H and a display image generation unit 19I are provided instead of the acquisition unit 18H and the display image generation unit 18I.

The transmission/reception unit 18A receives the analysis result information from the server device 11 as a response to the analysis result acquisition request transmitted to the server device 11, as in the first embodiment. In the present embodiment, the analysis result information includes the image ID, the omnidirectional panoramic image 60 corresponding to the image ID, and the person M specified for each person region 62 included in the omnidirectional panoramic image 60. The position coordinates indicating the first position and the position coordinates indicating the second position are included.

The acquisition unit 19H acquires this analysis result information. Specifically, the acquisition unit 19H corresponds to each of the representative image as the omnidirectional panoramic image 60 to be displayed and the representative image and each of the one or more omnidirectional panoramic images 60 to be reflected in the representative image. Get analysis information. Information indicating which of the plurality of spherical panoramic images 60 is the representative image and information indicating which is the image to be reflected on the representative image is acquired from the input unit 18F. You may. That is, the user inputs which of the plurality of spherical panoramic images 60 is to be displayed and which is to be the target of reflecting the position of the person M in the representative image. Then, the acquisition unit 19H may acquire these pieces of information from the input unit 18F via the reception unit 18C.

The display image generation unit 19I generates the display image 72. In the present embodiment, the display image generation unit 19I includes a first drawing unit 19J and a second drawing unit 19K.

The first drawing unit 19J draws the drawing point B at the first position specified in the person area 62 of the representative image in the omnidirectional panoramic image 60 designated as the representative image to be displayed.

The second drawing unit 19K selects the first position or the first position specified in the person area 62 of the other spherical panorama image 60 to be reflected in the spherical panorama image 60 designated as the representative image to be displayed. The drawing point B is drawn at the position 2.

In detail, the second drawing unit 19K features the peripheral area around the human area 62 in the omnidirectional panorama image 60 to be reflected and the corresponding area in the omnidirectional panorama image 60 as a representative image. When the amounts match, the drawing point B is drawn at the first position specified by the person region 62 in the spherical panoramic image 60 as the representative image.

The fact that the feature amounts of the peripheral region match means that the subjects in the peripheral region are the same. Specifically, if the feature amounts of the peripheral areas are the same, it means that the peripheral object D of the peripheral area is the same object.

For example, it is assumed that the peripheral object D of a certain person area 62 in the omnidirectional panoramic image 60 to be reflected is a chair, and the chair exists at the corresponding position in the omnidirectional panoramic image 60 as a representative image. In this case, the second drawing unit 19K determines the first position (the posture of the person M) specified in the person region 62 of the omnidirectional panorama image 60 to be reflected in the omnidirectional panorama image 60 as the representative image. The drawing point B is drawn at a position based on

On the other hand, the second drawing unit 19K determines that the feature amounts of the peripheral area around the human area 62 in the omnidirectional panorama image 60 to be reflected and the corresponding area in the omnidirectional panorama image 60 as the representative image are If they do not match, the drawing point B is drawn at the second position specified by the person area 62 in the spherical panoramic image 60 as the representative image.

The fact that the feature amounts of the peripheral region do not match means that the subjects in the peripheral region are different. Specifically, the fact that the feature amounts of the peripheral areas do not match indicates that the peripheral object D of the peripheral area is different.

For example, it is assumed that the peripheral object D of a certain person area 62 in the omnidirectional panoramic image 60 to be reflected is a chair, and no chair exists at the corresponding position in the omnidirectional panoramic image 60 as a representative image. In this case, the second drawing unit 19K determines the second position (around the person M) specified in the person area 62 of the omnidirectional panorama image 60 to be reflected in the omnidirectional panorama image 60 as the representative image. A drawing point B is drawn at a position independent of the area).

As a result, the display image generation unit 19I generates the display image 72.

FIG. 24 is an explanatory diagram of the display image 72. For example, it is assumed that the omnidirectional panoramic image 60J and the omnidirectional panoramic image 60K are obtained by the imaging device 14 (see FIGS. 24A and 24B).

Then, by the position specifying unit 11Q, for each of the persons M (person M10, person M11, person M12) included in the omnidirectional panoramic image 60J, the position P10 ₁ , as the first position based on the posture of the person M, It is assumed that the position P11 ₁ and the position P12 ₁ are specified. Similarly, by the position specifying unit 11Q, for each of the persons M (person M10, person M11, person M12) included in the omnidirectional panorama image 60J, the peripheral object D (here, chair, peripheral object D10, peripheral area) of the person M is included. It is assumed that the position P′10 ₁ , the position P′11 ₁ , and the position P′12 ₁ are specified as the second position independent of the object D11 and the peripheral object D12).

Further, it is assumed by the position specifying unit 11Q, and the person M included in the full spherical panoramic image 60K (human M11), as the first position based on the posture of the person M11, position P'11 ₂ were identified. Similarly, regarding the person M (person M11) included in the omnidirectional panoramic image 60J, it is assumed that the peripheral object has not been specified by the position specifying unit 11Q.

It is assumed that the omnidirectional panoramic image 60J is designated as the representative image to be displayed and the omnidirectional panoramic image 60K is designated as the image to be reflected. In this case, the display image generation unit 19I generates the display image 72J shown in FIG.

Specifically, the display image generation unit 19I causes the first position of each of the persons M (person M10, person M11, person M12) included in the omnidirectional panorama image 60J on the omnidirectional panorama image 60J ( The drawing point B is drawn at the position P10 ₁ , the position P11 ₁ , and the position P12 ₁ .

In addition, the display image generation unit 19I compares the peripheral object of the person M11 included in the spherical panoramic image 60K with the peripheral object D11 at the corresponding position in the spherical panoramic image 60J. In the example shown in FIG. 24, the peripheral object D does not exist in the person M11 of the omnidirectional panoramic image 60K, but the peripheral object D11 exists in the person M of the omnidirectional panoramic image 60J. In this case, the display image generation unit 19I determines that the feature amount of the peripheral area of the person M11 in the omnidirectional panoramic image 60K to be reflected and the corresponding area of the omnidirectional panoramic image 60J as the representative image do not match. to decide. Therefore, the display image generation unit 19I draws the drawing point B′ at the second position (position P′11 ₂ ) of the person M11 of the omnidirectional panorama image 60K on the omnidirectional panorama image 60J. As a result, the display image generation unit 19I creates the display image 72J shown in FIG.

On the other hand, it is assumed that the omnidirectional panoramic image 60K is designated as the representative image to be displayed and the omnidirectional panoramic image 60J is designated as the image to be reflected. In this case, the display image generation unit 19I generates the display image 72K shown in FIG.

Specifically, the display image generation unit 19I draws the person M (person M11) included in the omnidirectional panorama image 60K at the first position (position P11 ₂ ) on the omnidirectional panorama image 60K. Draw point B.

Then, the display image generation unit 19I compares the peripheral object D10 (chair) of the person M10 included in the spherical panoramic image 60J with the peripheral object D10 (chair) at the corresponding position in the spherical panoramic image 60K. To do. In the example shown in FIG. 24, these peripheral objects D10 match. In this case, the display image generation unit 19I determines that the peripheral object D of the person M10 in the omnidirectional panoramic image 60J to be reflected and the corresponding area in the omnidirectional panoramic image 60K of the representative image have the same feature amount. .. Therefore, the display image generation unit 19I draws the drawing point B at the first position (position P10 ₁ ) of the person M10 in the omnidirectional panorama image 60J on the omnidirectional panorama image 60K.

In addition, the display image generation unit 19I compares the peripheral object D11 (chair) of the person M11 included in the omnidirectional panoramic image 60J with the corresponding position in the omnidirectional panoramic image 60K. In the example shown in FIG. 24, the peripheral object D corresponding to the peripheral object D11 does not exist in the spherical panoramic image 60K. Therefore, in this case, the display image generation unit 19I does not match the feature amount between the peripheral object D11 of the person M11 in the omnidirectional panoramic image 60J to be reflected and the corresponding area in the omnidirectional panoramic image 60K of the representative image. It is determined that Therefore, the display image generation unit 19I draws the drawing point B′ at the second position (position P′11 ₁ ) of the person M11 of the omnidirectional panorama image 60J on the omnidirectional panorama image 60K.

In addition, the display image generation unit 19I determines that the person M12 included in the spherical panoramic image 60J is not present at the corresponding position in the spherical panoramic image 60K. In this case, the display image generation unit 19I draws the drawing point B′ at the second position (position P′12 ₁ ) of the person M12 of the omnidirectional panorama image 60J on the omnidirectional panorama image 60K. As a result, the display image generation unit 19I creates the display image 72J shown in FIG.

Note that FIG. 24 illustrates, as an example, the case where the number of the target omnidirectional panoramic image 60 reflected in the omnidirectional panoramic image 60 as the representative image is one. However, there may be a plurality of target spherical panoramic images 60 that are reflected in the spherical panoramic image 60 as the representative image. The display image generation unit 19I may generate the display image 72 in the same manner even in the case of a plurality of images.

Further, in the present embodiment, the case where the information processing terminal 19 is configured to include the acquisition unit 19H and the display image generation unit 19I has been described. However, the control unit 11A of the server device 11 may include the acquisition unit 19H and the display image generation unit 19I.

Next, a procedure of information processing executed by the server device 11 will be described. FIG. 25 is a flowchart showing an example of an information processing procedure executed by the control unit 11A of the server device 11.

11 A of control parts perform the process of step S300-step S308 similarly to step S100-step S110 which the control part 12A of 1st Embodiment performs.

Next, 11 A of control parts repeat the process of step S310-step S336 about the person area 62 detected by step S308 about the omnidirectional panoramic image 60 received by step S300.

In detail, the periphery specifying unit 12R specifies the peripheral object D around the person area 62 (step S310). Next, the periphery specifying unit 12R determines whether or not the peripheral object D has been specified in step S310 (step S312). When a negative determination is made in step S312 (step S312: No), the process proceeds to step S320 described below. If an affirmative decision is made in step S312 (step S312: Yes), the operation proceeds to step S314.

In step S314, the position specifying unit 11Q specifies the second position in the person area 62 (step S314). Next, the position specifying unit 11Q registers the specified second position in the image management DB 11C via the storage control unit 11P (step S316).

Next, the peripheral identification unit 12R determines whether the peripheral object D identified in step S310 is superimposed or non-superimposed on the person M in the processing target person area 62 (step S318). When it is determined that the peripheral object D is not superposed on the person M (step S318: No), the process proceeds to step S320.

On the other hand, when it is determined that the peripheral object D is superimposed on the person M (step S318: Yes), the process proceeds to step S322. In step S322, the moving body determination unit 12X determines whether or not the peripheral object D identified in step S310 is a moving body (step S322).

When it is determined that the peripheral object D superimposed on the person M is a moving body (step S322: Yes), the process proceeds to step S320.

In step S320, the posture determination unit 12J determines the posture of the person M (step S320). Next, the posture determination unit 12J determines whether or not the posture of the person M could be determined (step S324). When the posture of the person M can be determined (step S324: Yes), the process proceeds to step S326.

In step S326, it is determined whether the peripheral object D has been identified by the process of step S310 (step S326). When the peripheral object D can be specified (step S326: Yes), the process proceeds to step S328.

In step S328, the position specifying unit 11Q specifies the first position of the person M from the posture of the person M and the peripheral object D (step S328). Then, the process proceeds to step S336.

On the other hand, if it is determined in step S326 that the peripheral object D could not be specified (step S326: No), the process proceeds to step S330. In step S330, the position specifying unit 11Q specifies the first position of the person M from the posture of the person M (step S330). Then, the process proceeds to step S336.

On the other hand, when it is determined in step S324 that the posture of the person M cannot be determined (step S324: No), the process proceeds to step S332. In step S332, the position specifying unit 11Q specifies the predetermined position in the person area 62 as the first position of the person M (step S332). Then, the process proceeds to step S336.

Furthermore, when it is determined in the determination in step S322 that the peripheral object D superimposed on the person M is a non-moving object (step S322: No), the process proceeds to step S334. In step S334, the position specifying unit 11Q specifies the predetermined position of the person area 62 as the first position of the person M (step S334). Then, the process proceeds to step S336.

In step S336, the storage control unit 11P sets the position coordinates indicating the first position of the person M identified by the processing in steps S328 to S334 to the image ID of the omnidirectional panorama image 60 to be processed and the processing target. It is registered in the image management DB 11C in association with the person area ID of the person area 62 (step S336).

Then, the control unit 11A repeats the processes of steps S310 to S336 for all of the human area 62 detected in step S308 for the spherical panoramic image 60 received in step S300, and then ends the repetition. Then, this routine ends.

On the other hand, when a negative determination is made in step S300 (step S300: No), the process proceeds to step S338. Then, the control unit 11A executes the processes of steps S338 to S346 in the same manner as steps S134 to S142 of the control unit 12A of the first embodiment. Then, this routine ends.

Next, an example of a procedure of information processing executed by the information processing terminal 19 will be described. FIG. 26 is a flowchart showing an example of an information processing procedure executed by the information processing terminal 19.

First, the reception unit 18C of the information processing terminal 19 receives the analysis result acquisition request (step S400). Next, the transmission/reception unit 18A transmits the analysis result acquisition request accepted in step S400 to the server device 11 (step S402).

Next, the acquisition unit 19H acquires the analysis result information from the server device 11 via the transmission/reception unit 18A (step S404). Next, the first drawing unit 19J of the display image generation unit 19I sets the first position specified in the person area 62 of the representative image in the omnidirectional panoramic image 60 designated as the representative image to be displayed. , Drawing point B is drawn (step S406).

Next, the second drawing unit 19K identifies the first panorama image 60 specified as the representative image to be displayed, which is specified in the person region 62 of the other panorama image 60 to be reflected. The drawing point B or the drawing point B′ is drawn at the position or the second position (step S408). As a result, the display image generation unit 19I generates the display image 72.

Next, the display control unit 18D displays the generated display image 72 on the display unit 18G (step S410). Then, this routine ends.

As described above, the position specifying unit 11Q of the server device 11 (information processing device) according to the present exemplary embodiment defines, for each detected person region 62, the first position as the position of the person M based on the posture. , A second position independent of the change in the peripheral area (peripheral object D) around the person area 62 is specified. Then, the position specifying unit 11Q generates analysis information including the first position and the second position specified for each person region 62 included in the omnidirectional panoramic image 60 (image).

Therefore, in addition to the effects of the first embodiment, the server device 11 of the present embodiment has a position (second position) that is independent of the change in the peripheral region for each person M. A position (first position) based on the posture can be accurately specified.

In addition, the acquisition unit 19H uses the representative image as the image to be displayed (the spherical panoramic image 60), and the representative image and one or more images to be reflected on the representative image (the spherical panoramic image 60). And analysis information corresponding to each of. The display image generation unit 19I includes a first drawing unit 19J and a second drawing unit 19K. The first drawing unit 19J draws a first drawing point at a first position in the representative image specified by the person area 62 of the representative image. The second drawing unit 19K draws a drawing point at the first position or the second position specified in the person area 62 of the image to be reflected (the omnidirectional panoramic image 60) in the representative image. Then, the second drawing unit 19K, when the peripheral region (peripheral object D) around the person region 62 in the reflection target image and the corresponding region in the representative image have the same feature amount, A drawing point is drawn at the first position specified in the area 62. In addition, the second drawing unit 19K determines that the feature amounts of the peripheral area (peripheral object D) around the human area 62 in the image to be reflected (the spherical panoramic image 60) and the corresponding area in the representative image do not match. If there is, or if the person area 62 is not included in the representative image, a drawing point is drawn at the second position specified by the person area 62 in the representative image.

The programs for executing the above processes executed by each of the server device 12, the server device 11, the information processing terminal 18, and the information processing terminal 19 are NV-RAM, ROM, and other non-volatile storage media. It is provided by being pre-installed in. Further, a program for executing the above-described processing, which is executed by each of the server device 12, the server device 11, the information processing terminal 18, and the information processing terminal 19, is a file in an installable format or an executable format on a CD. It is also possible to record and provide in a computer-readable recording medium such as a ROM, a flexible disk (FD), a CD-R, a DVD (Digital Versatile Disk).

Further, a program for executing the above-described processing, which is executed by each of the server device 12, the server device 11, the information processing terminal 18, and the information processing terminal 19, is stored on a computer connected to a network such as the Internet. , May be provided or distributed by being downloaded via a network.

Further, the server device 12, the server device 11, the information processing terminal 18, and the information processing terminal 19 are each configured to be provided with a program for executing the above-described processing pre-installed in a ROM or the like. Good.

In addition, the program for executing the above-described processing, which is executed by each of the server device 12, the server device 11, the information processing terminal 18, and the information processing terminal 19, has a module configuration including the above-described units. As hardware, a CPU (processor) reads the program from a storage medium and executes the program to load the above-mentioned units onto the main storage device and generate the above-mentioned units onto the main storage device.

The various information stored in the storage units (storage unit 12B, storage unit 11B, storage unit 18E) of the server device 12, the server device 11, the information processing terminal 18, and the information processing terminal 19 are stored in an external device. May be. In this case, the external device and the server device 12, the server device 11, the information processing terminal 18, and the information processing terminal 19 may be connected via a network or the like.

Although the present embodiment has been described above, the above embodiment is presented as an example and is not intended to limit the scope of the invention. The novel embodiment can be implemented in various other forms, and various omissions, replacements, and changes can be made without departing from the spirit of the invention. The above-described embodiments and modifications thereof are included in the scope and the gist of the invention, and are also included in the invention described in the claims and the scope equivalent thereto.

10, 10A Information processing system 11, 12 Server device 11Q, 12Q Position specifying unit 12J Posture determining unit 12R Surrounding specifying unit 12T Superimposition determining unit 12X Mobile body determining unit 18, 19 Information processing terminal 18H Acquisition unit 18I, 19I Display image generation unit 19J 1st drawing part 19K 2nd drawing part

JP, 2005-087841, A

Claims (14)

A detection unit that detects a person area included in the image,
A posture determination unit that determines the posture of the person shown in the person region,
A position specifying unit that specifies the position of the person in the image based on the posture;
A superimposition determining unit that determines whether another object is superimposed or non-superimposed on the person shown in the person region in the image,
A moving body discriminating unit which discriminates whether the other object is a moving body or a non-moving body,
The position specifying unit,
When it is determined that another object is superposed on the person shown in the person area, and when the other object is a non-moving object, the predetermined position in the person area is set to Identified as a position,
If it is determined that another object is superimposed on the person shown in the person area, and if it is determined that the other object is a moving body, then it is determined based on the posture of the person shown in the person area. Locate the person in the image,
When it is determined that another object is not superposed on the person shown in the person area, the position of the person in the image is specified based on the posture,
Information processing apparatus. A detection unit that detects a person area included in the image,
A posture determination unit that determines the posture of the person shown in the person region,
A position specifying unit that specifies the position of the person in the image based on the posture;
A display image generation unit that generates a display image in which a drawing point is drawn at the position of the identified person in the image to be displayed,
The position specifying unit,
For each of the detected person regions, a first position as the position of the person based on the posture, and a second position that is independent of changes in peripheral regions around the person region are specified,
Generating analysis information including the first position and the second position specified for each of the person regions included in the image,
Information processing device. A detection unit that detects a person region included in a omnidirectional panorama image, which is an omnidirectional panorama image obtained by shooting in an omnidirectional range,
A horizontal correction unit that corrects the horizontal direction in the spherical panoramic image to match the horizontal direction in the shooting environment;
A distortion correction unit that corrects the distortion of the spherical panoramic image,
A posture determination unit that determines the posture of a person included in the corrected spherical panoramic image,
A position specifying unit that specifies the position of the person in the spherical panoramic image based on the posture;
An information processing apparatus including. A peripheral specifying unit that specifies a peripheral area around the person area in the image,
The position specifying unit,
Specifying the position of the person in the image based on the posture and the peripheral region,
The information processing apparatus according to any one of claims 1 to 3 . The position specifying unit,
When the posture of the person shown in the person area is not determined, a predetermined position in the person area is specified as the position of the person,
The information processing apparatus according to any one of claims 1 to 4. A display image generation unit that generates a display image in which drawing points are drawn at the position of the specified person in the image to be displayed,
The information processing device according to any one of claims 1 to 5 . A representative image as the image to be displayed, and an acquisition unit that acquires the representative image and the analysis information corresponding to each of the one or more images to be reflected on the representative image,
The display image generation unit,
A first drawing unit that draws a first drawing point at the first position specified in the person area of the representative image in the representative image;
A second drawing unit that draws a drawing point at the first position or the second position specified in the person region of the image to be reflected in the representative image,
The second drawing unit is
When the feature amounts of the peripheral area around the person area in the image to be reflected and the corresponding area in the representative image match, the characteristic image is located at the first position specified in the person area in the representative image. Draw a drawing point,
If the feature amount of the peripheral area around the person area in the image to be reflected and the corresponding area in the representative image do not match, or if the person area is not included in the representative image, the representative image , Drawing a drawing point at the second position specified in the person area,
The information processing apparatus according to claim 2 . The position specifying unit,
In the specification information in which the posture and the position specifying condition for specifying the position of the person are associated, the position specified by the position specifying condition corresponding to the determined posture is specified as the position of the person. The information processing apparatus according to any one of claims 1 to 7. The position specifying unit,
The position specifying condition corresponding to the determined posture and the specified peripheral region in the specifying information in which the posture, the peripheral region, and the position specifying condition for specifying the position of the person are associated with each other. The information processing apparatus according to claim 4 , wherein the position specified by is specified as the position of the person. A reception unit that receives an instruction to change the specific information,
A change unit for changing the specific information according to the change instruction,
The information processing apparatus according to claim 9, comprising: The information processing device according to claim 1 , wherein the image is a spherical panoramic image obtained by photographing in an omnidirectional range. An information processing system comprising an information processing device and an information processing terminal communicating with the information processing device,
The information processing device,
A detection unit that detects a person area included in the image,
A posture determination unit that determines the posture of the person shown in the person region,
A position specifying unit that specifies the position of the person in the image based on the posture;
A superimposition determining unit that determines whether another object is superimposed or non-superimposed on the person shown in the person region in the image,
A moving body discriminating unit which discriminates whether the other object is a moving body or a non-moving body,
The position specifying unit,
When it is determined that another object is superposed on the person shown in the person area, and when the other object is a non-moving object, the predetermined position in the person area is set to Identified as a position,
If it is determined that another object is superimposed on the person shown in the person area, and if it is determined that the other object is a moving body, then it is determined based on the posture of the person shown in the person area. Locate the person in the image,
When it is determined that other objects are not superposed on the person shown in the person area, the position of the person in the image is specified based on the posture,
The information processing terminal,
A display control unit that displays a display image showing the position of the person on the display unit;
Information processing system. An information processing system comprising an information processing device and an information processing terminal communicating with the information processing device,
The information processing device,
A detection unit that detects a person area included in the image,
A posture determination unit that determines the posture of the person shown in the person region,
A position specifying unit that specifies the position of the person in the image based on the posture;
A display image generation unit that generates a display image in which a drawing point is drawn at the position of the identified person in the image to be displayed,
The position specifying unit,
For each of the detected person regions, a first position as the position of the person based on the posture, and a second position that is independent of changes in peripheral regions around the person region are specified,
Generating analysis information including the first position and the second position specified for each of the person regions included in the image,
The information processing terminal,
A display control unit that displays a display image showing the position of the person on the display unit;
Information processing system. An information processing system comprising an information processing device and an information processing terminal communicating with the information processing device,
The information processing device,
A detection unit that detects a person region included in a omnidirectional panoramic image obtained by shooting in an omnidirectional range,
A horizontal correction unit that corrects the horizontal direction in the spherical panoramic image to match the horizontal direction in the shooting environment;
A distortion correction unit that corrects the distortion of the spherical panoramic image,
A posture determination unit that determines the posture of a person included in the corrected spherical panoramic image,
A position specifying unit that specifies the position of the person in the omnidirectional panoramic image based on the posture;
The information processing terminal,
A display control unit that displays a display image showing the position of the person on the display unit;
Information processing system.