JP5054753B2 - Information processing apparatus and information processing method - Google Patents

Description

  The present invention relates to information processing technology, and more particularly to an information processing apparatus that provides information using position information of a person and an information processing method performed there.

  Currently, various services using position information obtained from a global positioning system (GPS) are provided in car navigation systems, mobile phones, and the like. For example, an apparatus that maps and displays a history of a user's position on a map is also provided. A technique for further analyzing the movement history and grasping the behavior tendency of the user has also been proposed. For example, a method has been proposed in which the concept of “location” having a spread is introduced by clustering movement histories by the k-means method (see, for example, Non-Patent Document 1).

"Platform for associating personal contents and places", IPSJ Research Report, UBI, [Ubiquitous Computing System], 2007 (118) pp.9-14, 20071129

  When using the movement history in which position information such as latitude and longitude obtained from GPS or the like is associated with time, as described above, such as mapping on a map, information as a movement route can be obtained. Since the data is handled in the same row, it is not suitable for grasping the trend. In addition, there is a problem that the amount of data tends to be enormous and the display becomes complicated or difficult to search. Also, with the above-described clustering technique, although “location” information is extracted, the relationship with the time axis such as the stay time and the number of stays cannot be obtained. In addition, since each data is classified into “stop” and “movement” based on the moving speed, it is difficult to extract a staying place with movement such as an amusement park or a golf course.

  The present invention has been made in view of such a problem, and an object thereof is to provide an information processing technique capable of efficiently extracting useful information using a time history of position information.

  One embodiment of the present invention relates to an information processing device. This information processing apparatus obtains a time change of a parameter representing a speed from a position history recorded by associating a position coordinate of an object with a time, and a speed lower than a predetermined speed threshold is a predetermined duration threshold By extracting the period that has been continued as a stay period, a classification unit that classifies the time axis into a stay section and a movement section and classifies the stay locations corresponding to a plurality of stay sections classified by the classification unit. Generates travel history information that associates the stay location specified by the stay location specifying unit and the location of the stay specified by the stay location specifying unit with the corresponding stay section on the time axis and stores it in the travel history storage unit The movement history information forming unit to be used and the movement history satisfying the condition derived by the search key from the plurality of movement history information stored in the movement history storage unit using the information related to the place as the search key A search unit for searching for broadcast, wherein the search unit equipped with a behavior information generating unit to visualize the extracted movement history information in a predetermined form as a search result.

  Another aspect of the present invention relates to an information providing system. The information providing system includes a server connected to an information processing apparatus operated by a user via a network, and the information processing apparatus represents a speed from a position history in which position coordinates of an object are recorded in association with time. By classifying the time axis as a stay period and a travel period by extracting the period during which the speed below the predetermined speed threshold value continues for the predetermined duration threshold value or more as the stay period. The stay location identifying unit for identifying the location of the stay location corresponding to the plurality of stay sections classified by the classification unit from the distribution of the position coordinates in the stay segment, and the location of the stay location identified by the stay location identifying unit A movement history information forming unit that generates movement history information associated with the corresponding stay section on the time axis and transmits the movement history information to the server. A plurality of the movement history information from a device, a database generation unit for constructing a database of movement history information, and a condition derived from the database by the search key using the movement history information transmitted from the information processing device as a search key And a transmission unit that transmits the movement history information extracted as a search result by the search unit to the information processing apparatus in a predetermined format.

  Another aspect of the present invention relates to an information processing method. In this information processing method, a position history in which a position coordinate of an object is recorded in association with time is read from a memory, a time change of a parameter indicating speed is obtained, and a predetermined speed threshold is set in the change of the parameter indicating speed. By extracting the period during which the speed below the value has continued for a predetermined duration threshold or more as a stay period, the time axis is divided into a stay section and a movement section, and the classified multiple stay sections are supported. The location of the staying place to be identified from the position coordinate distribution in the staying section, and the movement history information relating the position of the identified staying place to the corresponding staying section on the time axis is generated and stored in the storage device The conditions derived by the search key from the plurality of movement history information stored in the storage device using the step and the information related to the location as the search key. Retrieving the plus movement history information, characterized in that it comprises the steps of: outputting to the output device to visualize the movement history information in a predetermined format extracted as a search result.

  Note that any combination of the above-described components, and the expression of the present invention converted between a method, an apparatus, a system, a computer program, a recording medium on which the computer program is recorded, and the like are also effective as an aspect of the present invention. .

  ADVANTAGE OF THE INVENTION According to this invention, various information based on a person's action can be provided using a positional information.

FIG. 3 is a diagram showing an outline of a data processing flow in the first embodiment. 1 is a diagram illustrating a configuration of an information processing device in Embodiment 1. FIG. FIG. 3 is a diagram showing a configuration of a position history analysis unit in the first embodiment in more detail. 6 is a diagram illustrating a specific example of a speed history calculated by a speed history calculation unit in Embodiment 1. FIG. It is a figure which shows the relationship of the stay area extracted when four speed threshold values are set in Embodiment 1. FIG. It is a figure which shows a mode that the integration part in Embodiment 1 integrates a stay section. 6 is a diagram for explaining a technique for integrating stay sections in Embodiment 1. FIG. It is a figure which shows a mode that it rotated so that the inclination of the ellipse which approximates the distribution of the position coordinate of each stay area in Embodiment 1 may become zero. 4 is a flowchart illustrating a procedure for analyzing a position history by a position history analysis unit in the first embodiment. 3 is a diagram illustrating in detail a configuration of a behavior information generation unit according to Embodiment 1. FIG. It is a figure which shows the example which made the movement history with respect to the user's one trip in Embodiment 1 into action information by imaging from a viewpoint of a stay place and a movement route. It is a figure which shows the example which made the movement history with respect to the user's one trip in Embodiment 1 into action information by imaging from a viewpoint of a stay place and a movement route. It is a figure which shows another example of the behavior information which a behavior information generation part produces | generates in Embodiment 1. FIG. It is a figure which shows another example of the behavior information which a behavior information generation part produces | generates in Embodiment 1. FIG. It is a figure which shows another example of the behavior information which a behavior information generation part produces | generates in Embodiment 1. FIG. 1 is a diagram illustrating a configuration of a system that provides information based on movement histories of a plurality of people in Embodiment 1. FIG. It is a figure which shows the structure of the action information generation part in Embodiment 2 in detail. It is a figure explaining the relationship between the tree diagram and display image in Embodiment 2. FIG. 10 is a flowchart illustrating a processing procedure when generating and displaying a display image for displaying a movement history in the second embodiment. 10 is a diagram showing an example of an image actually displayed in the second embodiment. FIG. 10 is a diagram showing an example of an image actually displayed in the second embodiment. FIG. 10 is a diagram showing an example of an image actually displayed in the second embodiment. FIG. FIG. 10 is a diagram for explaining a relationship between an image and a viewpoint when the images are integrated as one image and the viewpoint is switched in the second embodiment. It is a figure which shows the structure of the action information generation part in Embodiment 3 in detail. FIG. 10 is a diagram illustrating a method for determining the number of photos to be displayed for each section on the time axis in a movement history in the third embodiment. FIG. 10 is a diagram for describing a method for selecting a predetermined number of photos from photos taken in a certain section in a movement history in the third embodiment. 14 is a flowchart illustrating a processing procedure for performing a slide show with photos accompanied by movement history information in the third embodiment. FIG. 20 is a diagram illustrating an example of an image displayed in a mode in which a movement history and a slide show are displayed in synchronization in the third embodiment. FIG. 20 is a diagram illustrating an example of an image displayed in a mode in which a movement history and a slide show are displayed in synchronization in the third embodiment. FIG. 20 is a diagram illustrating an example of an image displayed in a mode in which a movement history and a slide show are displayed in synchronization in the third embodiment.

Embodiment 1
In this embodiment, a staying place and a moving route are specified based on a position history in which a person's position is recorded with respect to time. Various information based on human behavior is presented by taking into account the concept of expansive places and temporal connections. The information that can be provided includes information related to a user's individual movement history, as well as location information obtained as a result of integrating the movement histories of a plurality of users. In the following description, information on a place based on such a person's action is also expressed as action information. In this embodiment, “person” can be appropriately replaced with any moving object such as “animal” or “vehicle”.

  FIG. 1 shows an outline of the flow of data processing in the present embodiment. The flow of data processing in this embodiment is broadly divided into three stages: position history acquisition, movement history generation, and behavior information generation. In the position history acquisition process (S1), the position information of a person is recorded in real time to obtain a position history. Here, the position history is data in which a person's position coordinates are continuously recorded at a predetermined time interval for a predetermined time. The unit of the position history may be various, such as one day, one outing, one trip, etc., and is specified by the device or the user. One unit of position history 2 is generated by the position history acquisition process of S1.

  In the movement history generation process (S2), the position history 2 is analyzed, and a simple sequence of position coordinates of the position history 2 such as a stay place, a movement route, and the number of visits is given a meaning as a human action, and classified and integrated. As a result, a movement history 4 is generated. The unit of the movement history can be various as well as the position history. In the behavior information generation process (S3), the movement history 4 is visualized as the behavior information 7 in a user-desired format. Depending on the format of the behavior information 7, a plurality of units of movement history 6 are used as input data. The processing from S1 to S3 may be executed in a continuous flow, or each output data may be stored in a storage device and executed individually according to a user instruction or the like. Further, the former two of the three stages or only the latter two may be executed continuously.

  The position history acquisition process in S1 can be realized using a satellite or the like using a general device or method such as a mobile phone or a car navigation system having a GPS reception function, and thus description thereof is omitted. Hereinafter, the movement history generation process (S2) and the behavior information generation process (S3) will be described.

  FIG. 2 shows the configuration of the information processing apparatus 10 in the present embodiment. The information processing apparatus 10 includes an instruction input reception unit 12 that receives an instruction input from a user, a position history analysis unit 14 that analyzes a position history and generates a movement history, and an action information generation unit 16 that generates action information based on the movement history. The output unit 18 for outputting the behavior information is included. Furthermore, the information processing apparatus 10 stores a position history storage unit 20 that stores a position history that is an analysis target of the position history analysis unit 14, an analysis condition storage unit 22 that stores various parameters used for analysis, and a movement history that is an analysis result. The movement history storage unit 24 is included.

  In FIG. 2 and the like, each element described as a functional block for performing various processes can be configured with a CPU (Central Processing Unit), a memory, and other LSIs in terms of hardware, and in terms of software, This is realized by a program loaded in the memory. Therefore, it is understood by those skilled in the art that these functional blocks can be realized in various forms by hardware only, software only, or a combination thereof, and is not limited to any one.

  The instruction input receiving unit 12 receives an instruction relating to the designation of a file of position history to be analyzed and the presentation of behavior information from the user. The received information is appropriately supplied to the position history analysis unit 14 or the behavior information generation unit 16 depending on the content. The instruction input receiving unit 12 may be realized by a general input device such as a mouse, a keyboard, a touch panel, or a button.

  The position history analysis unit 14 reads out the designated position history file from the position history storage unit 20 and analyzes it using the parameters stored in the analysis condition storage unit 22. Specifically, the data sequence consisting of time and position coordinates is classified into stay sections and movement sections on the time axis, and by evaluating the original data sequence for each section, the time flow and the space of the stay place Generates movement history information considering the spread. The generated result is stored in the movement history storage unit 24.

  The behavior information generation unit 16 reads out the movement history generated by the position history analysis unit 14 from the movement history storage unit 24 and generates the behavior information in a format according to the content instructed by the user to the instruction input reception unit 12. As a form of behavior information, (1) a stay place and a movement route in a movement history of one unit or a plurality of units are represented by a graphic and visualized, and (2) a search that matches a condition from a plurality of movement histories is performed. 3) An example of presenting a trend of a place or route by integrating a plurality of units of movement history is conceivable. Specific examples will be described later.

  The output unit 18 outputs behavior information composed of image data and text data generated by the behavior information generation unit 16. The output unit 18 includes an output device such as a display device or a printer provided in the information processing apparatus 10 and its controller. As shown in FIG. 2, the information processing apparatus 10 is connected to the information providing server 8 via a network, acquires information necessary for generating behavior information, or requests a part of processing from the information providing server 8. May be. Although a specific example will be described later, in this aspect, the present embodiment is configured as an information processing system including the information processing apparatus 10 and the information providing server 8.

  FIG. 3 shows the configuration of the position history analysis unit 14 in more detail. The position history analysis unit 14 includes a position history acquisition unit 26, a speed history calculation unit 28, a classification unit 30, an integration unit 32, and an information formation unit 34. The position history acquisition unit 26 acquires information on the position history file designated from the instruction input reception unit 12 and reads the position history file from the position history storage unit 20. In the following description, the position coordinates recorded in the position history are expressed by (latitude, longitude). However, depending on the position history recording means, the same applies to other expression methods such as the x-coordinate and y-coordinate on the map. Can be applied.

  The speed history calculation unit 28 obtains the speed at each time as a speed history based on the position history. If some of the position coordinate data is missing for some reason, such as radio interference, the data interpolation is performed based on the previous and next position coordinates. Further, in order to exclude abnormal values, the obtained speed history is smoothed. The accuracy of the subsequent analysis is increased from these pre- and post-processing.

  The classification unit 30 classifies the time axis in the position history, and thus the position coordinate data corresponding to each time, into the stay section and the movement section by performing threshold determination on the speed history. Although details will be described later, in the present embodiment, a plurality of classification patterns are generated by setting a plurality of threshold values. As a result, an analysis result that takes into account the difference in the level of abstraction due to the difference in speed scale, such as movement by airplane, movement by car, and movement by walking, can be obtained.

  The integration unit 32 considers stays at the same place in stay sections at different timings by feeding back the time flow classified by the classification unit 30 into stay sections and movement sections based on speed to position coordinates in each section. Sections are detected, and the stay sections are integrated as necessary. Specifically, the distribution of position coordinates in each stay section is approximated by a figure, and whether to stay at the same place or to be regarded as a single stay is determined according to the degree of overlap and the continuity of time. By further evaluating the distribution of the position coordinates for the integrated stay section, the final stay section and movement section are determined with respect to the passage of time. Further, the integration unit 32 may receive designation of stay sections to be integrated from the user, and integrate the stay sections and the movement sections sandwiched between the stay sections accordingly.

  The information forming unit 34 evaluates a set of position coordinates and speed values in each stay section and movement section obtained in this way, and derives predetermined information for each section. Furthermore, if it is a stay at the same place even if it is another opportunity, the information of each stay area is put together and it is set as the information with respect to a place. The information items derived here will be exemplified later. The information forming unit 34 stores the obtained information as a movement history in the movement history storage unit 24.

  FIG. 4 shows a specific example of the speed history calculated by the speed history calculation unit 28. The speed history 40 is represented by a speed change with respect to a time change (horizontal axis). As described above, such a speed history is obtained by interpolation of position coordinates, speed calculation, and smoothing processing. Interpolation of position coordinates is performed by applying an existing technique such as linear interpolation or spline interpolation to a data missing portion in a line connecting two-dimensional coordinates represented by (latitude, longitude) in time order. The speed at each time is basically obtained by dividing the amount of change in position coordinates from the previous time by the time interval. At this time, when information is finally presented using a map of Mercator projection, parameters of (latitude, longitude) are converted into (x coordinate, y coordinate) on the map.

If position information is acquired at time..., T-1, t, t + 1,..., The latitude lo (t) and longitude la (t) at time t are expressed as two-dimensional coordinates in the Mercator projection, x (t ) And y (t) are converted as follows.
x (t) = lo (t) × π / 180.0f
y (t) = asinh (tan (la (t) × π / 180.0f)

  When the speed is obtained for each time, the abnormal value is excluded by performing a smoothing process on the time change. The smoothing process is executed by applying an existing method such as a median filter. A speed threshold value Vth and a duration threshold value Tth are set for the speed history 40 thus obtained. Then, the classifying unit 30 extracts a section where the state where the speed is lower than the speed threshold value Vth continues for the duration threshold value Tth or more as the stay section. In FIG. 4, five stay sections of “first stay”, “second stay”, “third stay”, “fourth stay”, and “fifth stay” are extracted. By introducing the duration threshold value Tth, it is possible to exclude a stagnation of a short time that is not desirable to be extracted as a stay, such as a signal waiting state, from the extraction target.

  A plurality of speed threshold values are set in advance and stored in the analysis condition storage unit 22 together with the duration threshold value Tth. Then, the classification unit 30 performs threshold determination in order from the larger speed threshold. FIG. 5 shows the relationship between stay sections extracted when four speed thresholds are set, and the horizontal direction represents the passage of time. In the figure, it is assumed that the first speed threshold value, the second speed threshold value, the third speed threshold value, and the fourth speed threshold value increase in this order.

  First, the threshold determination is performed as described above by applying the first speed threshold to the speed in the entire period 41 in which the position history of the analysis target is recorded, and the stay section 42a is extracted. Next, the threshold value determination is performed with respect to only the extracted speed of the stay section 42a with the next second speed threshold value, and the stay section 42b is extracted. Furthermore, threshold determination is performed with the third speed threshold only for the speed of the stay section 42b, and the stay section 42c is extracted. Similarly, threshold determination is performed on the stay section 42c with the fourth speed threshold value, and the stay section 42d is extracted.

  By such processing, four patterns of stay sections are generated. At this time, the upper stay section pattern having a large speed threshold value includes the lower stay section pattern having a small speed threshold value. The number of speed threshold values to be set is not limited to four. The reason why a plurality of speed threshold values are introduced in the present embodiment is to switch according to the information presenting the abstraction level of the staying place. For example, if you want to process location history when traveling by airplane in multiple countries, you can use the pattern of the stay section with respect to the speed threshold near the flight speed to determine the place of stay in the unit of visited country or city. Can show. On the other hand, if the pattern of the stay section with respect to the speed threshold value near the walking speed is used, the place of stay can be shown in units such as buildings visited at the time of walking in the visited places in each country.

  By setting the threshold in multiple steps from the flight speed to the walking speed with multiple speed thresholds, the level of abstraction of the place can be changed in multiple steps according to the difference in distance feeling due to moving means such as trains and cars Can do. In any case, as shown in FIG. 5, since the stay section is extracted with respect to the passage of time, the stagnant action of staying including the number of visits and the travel route to the stay section is included in the flow of time. And can be expressed as a movement history.

  The stay sections extracted as described above are integrated based on the original position history. FIG. 6 shows a state where the integration unit 32 integrates the stay sections. Similar to FIG. 5, the passage of time is shown in the horizontal direction. The upper part is the stay section pattern 44 before integration, and the lower part is the stay section pattern 46 after integration. In the figure, for convenience, the sections before and after the stay section are set as movement sections, but the final movement section is determined after the final stay section pattern is determined after integration. First, in the stay section pattern 44 before integration, “first stay” from time t1 to t2, “second stay” from time t3 to t4, and “third stay” from time t5 to t6 are extracted as stay sections. To do.

  The stay section extraction performed by the classification unit 30 is based only on the speed. On the other hand, if you are staying in a place that is considered to be one place, such as a park, amusement park, golf course, or stadium, even if you run in it or ride a cart, It may be desirable to see it as a stay in one place. Therefore, the stay section extraction result by speed is fed back to the position coordinate information, so that the stay section is evaluated from the viewpoint of space, and the stay sections are integrated if necessary or the stay is changed Record as stay at the place.

  Specifically, it is determined whether or not the stay is in the same place from the overlap of the distribution of position coordinates in each stay section, and there is a stay section in another place between those stay sections. If not, it is determined that it is a single stay at the place and the stay sections are integrated. In the example of FIG. 6, “first stay”, “second stay”, and “third stay” are integrated into one stay section “first stay” from time t1 to time t6. In the actual integration process, “first stay” and “second stay” may be integrated first, and the resulting stay section and “third stay” may be integrated. By using the set of position coordinates in the “first stay” after integration finally obtained, the spatial extent of the place can be specified.

  FIG. 7 is a diagram illustrating a method for integrating stay sections. As described above, the integration process uses the stay section pattern based on the speed history and the position history information that is the original data. The left diagram in FIG. 7 shows a movement history line 48 that connects position coordinates at each time in order of time. The parameters from t1 to t6 in the figure correspond to each time shown in FIG. That is, the dotted line between times t1 and t2 is the stay section “first stay”, the alternate long and short dash line between times t3 and t4 is the stay section “second stay”, and the two-dot chain line between times t5 and t6 is the stay section “first stay”. The sections extracted as “3 stays” are shown. The solid line is the other section as described above, and is considered to be a moving section before integration.

  In such a plane of the movement history line 48, the distribution of position coordinates included in each stay section is approximated by a predetermined figure. For example, a graphic including all position coordinates is derived. When the overlapping ratio between the figures exceeds a predetermined overlapping ratio threshold, it is determined that the stay section corresponding to the figure is staying at the same place. The overlap ratio threshold value is stored in the analysis condition storage unit 22 and read by the integration unit 32. In this case, since the abnormal value of the position coordinate greatly affects the figure to be obtained, it is desirable to perform a smoothing process on the position history data. As in the case of obtaining the speed history, coordinate conversion is performed as necessary.

The figure used for approximation is determined in advance from a rectangle or an ellipse. Compared with a rectangle, an ellipse is often closer to the distribution of actual position coordinates because there is no corner area. On the other hand, the rectangle is easy to calculate. Therefore, it is selected as appropriate from the viewpoint of accuracy required and calculation load. In the example of FIG. 7, ellipses 50a, 50b, and 50c are derived for the respective stay sections of “first stay”, “second stay”, and “third stay”. When applying an ellipse, first, the coordinates of the center of gravity are obtained by averaging the position coordinates included in each stay section. Then, the rotation angle of the ellipse is obtained as follows. First, when the barycentric coordinates are (m _x , m _y ), the secondary covariance matrix is given by the following equation.

  Here, (x (p), y (p)) represents each position coordinate. The eigenvalue λ of the matrix is calculated by the following equation.

  Then, using the eigenvalue λ, the inclination θ of the ellipse is obtained from the following equation.

Next, the position coordinates included in each stay section are rotated in the opposite direction to the inclination by the inclination θ of the ellipse around the center of gravity. FIG. 8 shows how the original position coordinate distribution is rotated so that the inclination of the ellipse becomes zero. The distribution 52 represents a point set of original position coordinates, and the distribution 54 represents a point set of position coordinates after rotation. In the distribution 54 thus obtained, the x coordinate of the point where the distance from the center of gravity (m _x , m _y ) is maximum in both directions of the x axis, that is, x _min , x _max and the maximum in both directions of the y axis. and a y-coordinate of the point where, i.e. _{y min,} and _{y max,} wherein the ellipse can be determined as follows.

Here, a = (x _max −x _min ) / 2 and b = (y _max −y _min ) / 2. Next, the overlapping ratio of the areas of the figures corresponding to the stay section obtained in this way is obtained. For example, a pixel inside / outside determination is performed by performing a raster scan on an image in which each ellipse is plotted. The number of pixels inside the figure and common to the plurality of figures corresponds to the area of the overlapping portion. Or you may logically calculate from the numerical formula showing figures, such as an ellipse and a rectangle. And when the ratio of the part of the figure corresponding to a certain stay section overlaps with another figure exceeds a predetermined overlap ratio threshold, the stay sections corresponding to the two figures are stays at the same place. Judge that there is.

  Here, when these stay sections are continuous stays with only the moving section sandwiched without staying at another place, the stay sections are integrated as shown in FIG. 6 to make one stay. On the other hand, when the visit to another place is sandwiched, it is assumed that the stay is another opportunity at the same place. When the stay sections are integrated, the distribution of the position coordinates in the stay sections after integration is approximated with a graphic in the same manner as described above, and the overlapping ratio with the graphic corresponding to the other stay section is obtained. By repeating this process for all stay sections, the stay section pattern can be accurately acquired, and the action “stay” can be associated with “place”.

  Next, an operation for generating a movement history according to the configuration described so far will be described. FIG. 9 is a flowchart showing a procedure for the position history analysis unit 14 to analyze the position history. First, when the instruction input receiving unit 12 receives an analysis instruction from the user, the position history acquisition unit 26 reads the specified position history file from the position history storage unit 20 (S10). Then, the speed history calculation unit 28 interpolates the missing portion of the data as necessary, calculates the speed at each time, and performs the smoothing process to obtain the speed history (S12).

  Next, the classification unit 30 reads a plurality of speed threshold values and duration threshold values from the analysis condition storage unit 22, sets the largest speed threshold value, performs threshold determination, and extracts a stay section. Thus, the stay section pattern is derived (S14, S16). The integration unit 32 acquires the position coordinates included in each stay section in the stay section pattern from the position history, and approximates the distribution with a graphic (S18). Then, after reading the overlap ratio threshold value from the analysis condition storage unit 22 and detecting a figure whose overlap ratio exceeds the threshold value (Y in S20), another stay is made between the stay sections corresponding to the figure. It is confirmed whether or not there is a continuous section (S22).

  If it is continuous (Y in S22), the stay sections are integrated into one stay section (S24). If it is not continuous (N in S22), the stay section is left as it is, and it is recorded that the stay is in the same place (S26). The processes from S18 to S26 are repeated for all stay sections (N in S28, S18, S20, S22, S24, S26). If the stay sections are integrated in S24, a new integration determination is also made for the stay sections after integration. When all the stay sections are determined (Y in S28), the stay sections are extracted with the next largest speed threshold value, and integrated determination is performed (N, S14, S16, 18, S20, S22, S24, S30, S26, S28). When the above process is repeated for all speed thresholds, the process ends (Y in S30).

  Then, the information forming unit 34 uses a plurality of stay section patterns corresponding to each speed threshold value, the original position history, the speed history acquired in the middle, and the like obtained in the above-described processing, to determine predetermined information. An arithmetic process corresponding to the item is performed, and final movement history information is generated. The following are examples of information items to be generated.

  First, as main items, the total time of analysis objects including stay and movement, the number of stay places, stay data for each place such as “place A” and “place B” are included. Moreover, the stay data for each visit such as the number of visits, the first time, and the second time are included as sub-items of the stay data for each place. Furthermore, as sub-items of stay data for each visit, the speed threshold used for the analysis, the upper place extracted by the upper speed threshold, the start time and end time of the stay, which were the targets in the location extraction Among the position coordinates recorded in the position history, the number of coordinates included in the place, the average speed, the average altitude, the average latitude, the average longitude, and the range of the place are included.

Here, the range of the place can be represented by the upper left coordinates of the rectangle and the length of the side when the distribution of the positions included in the stay section is approximated by a rectangle. If approximated by elliptical, the center of gravity _{(m x,} m _y), the major and minor axes of lengths (2a and 2b), can be expressed by the inclination angle theta. The average altitude can be calculated when the altitude at each time is recorded in the original position history. Since the value of each item can be calculated by a simple mathematical expression, a specific description is omitted.

More general information may be added to the personal movement history information obtained from the personal position history information as described above. For example, the information providing server 8 is connected to a server that provides a surrounding landscape photograph for any latitude and longitude, or a database that provides other information via a network, and is distributed according to the place of stay and location. Get the number of information you have. Then, calculate the degree of interest defined below for each location. Interest level = number of information / area of location

  That is, the degree of interest represents the density of information generally provided for a place, and is an index indicating the degree of public interest in the target place. By associating location-centric information obtained from many points of view from personal events to general trends in this way, information search efficiency and convenience through mashup display of various information are improved. To do. In particular, the present invention provides an unprecedented mode of information presentation and information retrieval by giving meaning to the enumeration of position coordinates given as a position history from the viewpoint of place, stay / movement action, and their temporal order. It becomes possible.

  The above table is merely an example, and information that can be acquired based on position coordinates by searching the database of the information processing apparatus 10 itself or the information providing server 8 such as a place name or a facility name is appropriately included in the movement history. Good. In the following description, data obtained by analyzing the position history is collectively referred to as movement history, and individual data such as one item is also referred to as movement history in some cases.

  Next, behavior information generated by the behavior information generation unit 16 based on the movement history obtained as described above will be described. FIG. 10 shows the configuration of the behavior information generation unit 16 in detail. The behavior information generation unit 16 includes a display image generation unit 130 that generates an image for movement history display and movement history search, a scale change instruction reception unit 131 that receives a scale change of the map when the movement history is displayed on the map, Classification pattern switching unit 132 that changes the stay / movement pattern of the movement history to be displayed according to the scale of the map, search key reception unit 133 that receives a search key input from the user, and a search unit that searches the movement history according to the search key 134.

  The behavior information generation unit 16 further stores data associated with any item of the movement history, such as a movement history graphic storage unit 135 that stores a graphic necessary for displaying the movement history, and photograph data and calendar graphic data. A stored additional information storage unit 136 and a scale / classification correspondence storage unit 137 that stores scale / classification correspondence information that associates a map scale range with a classification pattern to be used for switching the classification pattern are included.

  The display image generation unit 130 generates image data to be displayed as behavior information. For example, data such as a movement history display image representing a place of stay and a moving route on the map, and a search key input image and a search result display image when the movement history is searched using a search key are generated. The generated image data is output as a display image via the output unit 18. Data necessary for image generation is read from the movement history graphic storage unit 135 and the additional information storage unit 136. The movement history graphic storage unit 135 stores map data in addition to a graphic indicating a staying place, a graphic indicating a movement route, and the like. The user inputs an instruction to select the type of the image and, in some cases, a movement history, via the instruction input receiving unit 12, and the display image generation unit 130 generates display image data corresponding to the instruction.

  The scale change instruction accepting unit 131 accepts an instruction input by the user via the instruction input accepting unit 12 to change the scale of the map being displayed. The received information is supplied to the display image generation unit 130 to generate image data in which the scale of the map to be displayed is changed. Further, the same information is supplied to the classification pattern switching unit 132. The classification pattern switching unit 132 refers to the scale / classification correspondence information in the scale / classification correspondence storage unit 137 and determines whether or not the scale ratio exceeds a threshold value for switching the classification pattern. Although a specific example will be described later, in the present embodiment, a plurality of stay / movement section classification patterns are generated for a plurality of speed threshold values. By associating the range of the scale ratio to be displayed with the classification pattern in advance, the scale ratio of the map and the classification pattern to be displayed as the movement history are linked.

  When the classification pattern switching unit 132 determines to switch the classification pattern, the classification pattern switching unit 132 notifies the display image generation unit 130 to that effect. At this time, for example, the identification information of the classification pattern after switching is notified. The display image generation unit 130 switches the movement history display by newly reading out the movement history after switching from the movement history storage unit 24 and displaying it with a figure or the like.

  The search key reception unit 133 receives a search key input by the user via the instruction input reception unit 12 in a state where the search input image is displayed. The search key may be a character string or a cursor indicating a range on the map. Specific examples will be described later. When the search key is input, the search key receiving unit 133 supplies the search key to the search unit 134. The search unit 134 searches the movement history storage unit 24 based on the search key, and reads the corresponding movement history data. By giving the read data to the display image generation unit 130, the display image generation unit 130 generates data of an image displaying a movement history as a search result.

  The behavior information generation unit 16 does not have to include all of the above-described function of displaying a movement history, a function of switching a classification pattern according to a scale ratio, and a search function. A similar function may be provided in combination with a function module of another application. Next, a specific example of an image generated by the display image generation unit 130 will be described. FIG. 11 and FIG. 12 show an example in which the movement history for a single trip of the user is imaged from the viewpoint of the staying place and the moving route to be action information. In this case, the visibility of the movement history is improved by displaying the stay location and the movement route separately. Here, both the movement history display image 60a in FIG. 11 and the movement history display image 60b in FIG. 12 represent the movement history during the same trip, but as described above, the speed threshold for extracting the stay section is as follows. The value is different.

  In FIG. 11, by setting the speed threshold value to 200 km / h, it is possible to distinguish between movement and stay in a section traveling by airplane and other sections. On the other hand, in FIG. 12, by setting the speed threshold value to 20 km / h, it is possible to distinguish between movement and stay in a section moving by a vehicle and a section walking or stopping. Since the distance between the two standards is different, the scale of the distance to be displayed is changed. That is, the former speed threshold value is a country unit scale as in the movement history display image 60a in FIG. 11, and the latter speed threshold value is in units of individual facilities or roads as in the movement history display image 60b in FIG. It becomes the scale of.

  As described above, when the movement history is displayed as an image, the speed threshold used for extracting the stay section and the scale of the distance on the image are linked to dynamically abstract the user event. 11 and 12 show the movement history on the corresponding scale map, but the present embodiment is not limited to this. For example, when detailed information on the route is not necessary, the positional relationship between a plurality of staying places and the order of visiting may be indicated only by arrows. In either case, the speed threshold and the abstraction level of the staying place can be linked.

  The movement history display image 60a in FIG. 11 includes a movement history diagram 62a and a scale change lever 70. In the movement history diagram 62a, a place where the user stayed on the world map is represented by a double circle 64a, and a movement route between the two is represented by a line 66a. A circle 64a representing a staying place represents the total staying time of the staying place as a radius with the center of gravity of the distribution of position coordinates in the staying section as the center. Thereby, a time element can be added to the two-dimensional plane representing the position for display.

  A line 66a representing the movement route is obtained by connecting the position coordinates recorded in the movement section in time order. However, the display may be switched by connecting the staying places with a straight line depending on the situation where the detailed route is not important as shown in the movement history diagram 62a of FIG. In any case, the order of visits to the place of stay is clarified by representing the moving direction with an arrow. Further, the number of visits to each staying place is indicated by characters 68a. With this display, it is possible to grasp at a glance how many times and how many times a plurality of places have been visited.

  The scale change lever 70 is graphics for the user to adjust the scale of the map of the movement history diagram 62 a via the instruction input receiving unit 12. When the user changes the scale change lever 70 up and down with a pointing device or the like, the scale of the map of the movement history diagram 62a continuously changes. When the scale of the map increases and exceeds a predetermined threshold value, the movement history display image 60b of FIG. 12 is displayed.

  That is, the display is switched to the stay section patterns classified by different speed thresholds according to the threshold set for the scale ratio. For that purpose, the range of the scale ratio is associated with the speed threshold value in advance. Thereby, the movement history to be displayed seamlessly can be switched in conjunction with the change in the scale ratio. Of course, depending on the setting of the speed threshold value, the movement history to be displayed may be switched in multiple stages between FIG. 11 and FIG.

  The movement history display image 60b in FIG. 12 also includes a movement history chart 62b and a scale change lever 70. The movement history chart 62b includes a circle 64b representing a stay place, a line 66b representing a travel route, and the number of visits to the stay place. Is displayed. Since the movement history diagram 62b in FIG. 12 is a scale ratio at which roads are displayed as shown in the figure, it is clear which road has been passed during movement by representing the movement route as a line connecting the recorded position coordinates. Become.

  FIG. 13 shows another example of behavior information generated by the behavior information generation unit 16. In this example, a search screen for searching for a movement history that matches a condition from a plurality of units of movement history stored in the movement history storage unit 24 is displayed. The search target may be all movement histories stored in the movement history storage unit 24, or may be a movement history in which a position history is recorded within a period specified by the user. The example of FIG. 13 is an image representing a search result when a position on the map is used as a search key. Before entering this state, the behavior information generating unit 16 first displays a map 74 on the movement history search image 72. The map 74 allows the user to change the display range and scale via the instruction input receiving unit 12.

  Further, a cursor 76 that can be moved by the user via the instruction input receiving unit 12 is displayed on the map 74. The cursor 76 is circular so that a predetermined range on the map can be designated. With this cursor 76, the user designates a position to be a search key. Specifically, when the cursor 76 is moved to a desired place and a decision input is made by pressing the decision button of the instruction input receiving unit 12 or the like, the behavior information generation unit 16 sets the stay location within the range of the cursor 76. The included movement history is read from the movement history storage unit 24.

  Then, in the same manner as shown in FIGS. 11 and 12, based on the read movement history, the place of stay and the route included in the movement history are graphically displayed on the map 74. At this time, if a plurality of movement histories are applicable, they are distinguished by changing the color or shape of the figure. Further, predetermined items of the movement history items are displayed as character information 78. In FIG. 13, the date and time, the total time, and the number of places of stay where the position history that is the basis of the movement history is displayed are displayed as legends of the graphic representing the route of the movement history.

  By making the movement history searchable in this way, even a user who has recorded a huge position history in the past can easily find out the movement history staying at a certain place. For example, when planning a trip, you can search for the history of trips to the same direction in the past and acquire information such as the route of travel at that time and the store you visited. It is done. The movement history may not be displayed on the map, may be displayed on an illustration or aerial photograph on which the store is drawn, or the movement route may be displayed alone. In addition, only the distribution of staying places representing the staying places with color shading or graphics may be displayed.

  FIG. 13 illustrates the case where the position on the map is used as a search key. As described above, in this embodiment, the staying place is extracted as one piece of the movement history information. Therefore, the place can be the movement history and eventually the metadata of the individual event. FIG. 14 is a diagram for explaining another example of an image for realizing such an aspect. In this example, the movement history stored in the movement history storage unit 24 is collected for each date when the corresponding position history is recorded, and data obtained by adding photograph data taken on each day as additional information is used. In addition to photos, image data such as moving images and pictures acquired on each day and audio data such as music may be used as additional information.

  The movement history search image 80 of FIG. 14 includes a calendar 82 in which dates are expanded in squares, and as shown in the “October” field 83, thumbnails of the photos taken on that date are displayed as thumbnails. (Shaded rectangle in the figure). You may display a photograph only in the month which the user pointed through the instruction input reception part 12. FIG. Then, after selecting a certain month, when the user selects the item “search by place” from the menu selection window 84 of the movement history search image 80, a search key input window 86 is displayed. As shown in the figure, the search key input window 86 is an area for the user to input the latitude and longitude of the center position of the place as the search key and the range (radius) as a character string. This input information performs the same function as the cursor 76 in FIG. 13, and when the user inputs an instruction to execute a search, the movement history graphic and various information similar to those in FIG. 13 are displayed. However, it is assumed that the movement history to be searched is only recorded in the month selected by the calendar 82.

  In the case of FIG. 14, the user can narrow down the movement history to be searched by checking the photos displayed on the calendar 82. The user confirms the month of the event that the user wants to find out, finds the photo at the desired event in October, and enters the location information that is the search key with the month selected. To do. Since an image of a photograph is useful information for recalling human memory, it is possible to efficiently reach a desired movement history by narrowing down the movement history using the information. In addition, by associating with the date of the calendar, it is possible to provide a function as a diary or schedule management.

  FIG. 15 shows another example of behavior information generated by the behavior information generation unit 16. In this example, a plurality of unit movement histories of one user stored in the movement history storage unit 24 are integrated to generate new information related to the place. The integration target may be all the movement histories stored in the movement history storage unit 24, or a movement history in which a position history is recorded within a period specified by the user. Further, when the user selects a desired area on the map or by inputting a character string, the behavior information generating unit 16 reads a movement history including the staying location in the area from the movement history storage unit 24.

  Then, all the stay places and routes to the travel history of the read plural units are displayed on one image. In the example of the drop-in place display image 90 in FIG. 15, a map 92 of a region selected by the user, a graphic 94 representing a place where the user has stayed in the region, and a line 96 representing a route are displayed. If staying at the same place in the movement history of a plurality of units, only one graphic 94 is displayed. The line 96 representing the route is the same. Alternatively, a threshold may be set for the number of visits, and places that are not frequently visited and places that are frequently visited may be displayed in different colors. The stop location display image 90 in FIG. 15 alone is useful in the sense that it is possible to grasp a store visited in the past at a glance, but in this embodiment, further action information is generated using this information. .

  For example, information on stores in the corresponding area and its surroundings is acquired from the information providing server 8 connected via the network. The information shown in FIG. 15 is used to pick up stores that the user has not yet visited and display them in a list or mark them on a map. Alternatively, stores that have not been visited in the same category as stores that the user frequently visits are displayed in a list including the vicinity of the corresponding area or displayed on a map. You may consider frequently used routes for store pick-up. If information on a store that is frequently visited among stores that are frequently visited can be acquired, the graphic 94 is highlighted or character information is added to the drop-in place display image 90 in FIG.

  In this way, by acquiring and displaying information disclosed on the network in consideration of individual behavior patterns, it is possible to efficiently present only information that is likely to be useful to the user. The above is behavior information generation based on one person's movement history, but it can be applied to a plurality of people.

  FIG. 16 shows the configuration of a system that provides information based on the movement histories of a plurality of people. The information providing system 100 includes a plurality of information processing apparatuses 10a, 10b, 10c, and 10d and an information providing server 8. The information processing apparatuses 10a, 10b, 10c, and 10d may have the same configuration as the information processing apparatus 10 illustrated in FIG. Moreover, the number is not limited to that shown in the figure, and is, for example, the number of terminals operated by users participating in a social networking service (SNS). The information providing server 8 is a server that manages the community site, and includes a database generating unit 102, a database 120, and an information providing unit 104.

  The database generation unit 102 includes an individual information acquisition unit 106 that receives predetermined items of movement history transmitted from the plurality of information processing apparatuses 10a to 10d, and an information integration unit 108 that integrates the movement histories and stores them in the database 120. including. When the individual information acquisition unit 106 acquires information on a person's stay location and stay route as a movement history, and the information integration unit 108 integrates data of movement histories of a plurality of users for each area on the map, the same as FIG. Can be acquired from the movement history of a plurality of people. Information transmitted from the information processing apparatuses 10a to 10d is appended with actual situation information for each travel history such as travel, walk, commute, and the like. The information integration unit 108 integrates the transmitted movement histories and stores them in the database 120 for each situation.

  The information providing unit 104 receives a query related to the presentation of information transmitted from the information processing devices 10a to 10d, acquires necessary information from the database 120 according to the content of the query, and puts it in a predetermined format. A search unit 112 for shaping and a result transmission unit 114 for transmitting a result to the information processing apparatuses 10a to 10d that transmitted the inquiry are included. From the information processing apparatuses 10a to 10d, a signal for inquiry including information items that the user wants to acquire, such as area designation, situation designation, or recommended store, is transmitted.

  The search unit 112 searches the database 120 for movement history information corresponding to the region or situation included in the inquiry signal. This is the same information as shown in FIG. 15, but since it is based on the movement history of a plurality of people, it includes information such as popular shops and lesser known back roads. Therefore, by comparing with the personal movement history of the user transmitted from the inquiry source user, the user can present an unexplored but popular place. At the time of presentation, the above-described interest level information on the place may be added.

  As yet another example, traffic speed information at each time can be presented by calculating an average speed on a highway from a plurality of movement histories and classifying the average speed on a day of the week, a time zone, a season, and the like. For example, when the user transmits a route scheduled for a trip as an inquiry from the information processing apparatus 10a, the information providing server 8 transmits information on a traffic jam portion in the same route in the same situation. Further, a movement history including a route passing through the start point and the end point of a traffic jam location may be searched from the database 120, and the detour and its average speed may be transmitted.

  By accumulating the movement history of multiple people in this way and using it to provide information, it is difficult to understand by searching using names etc., and the degree of popularity of stores and information that could not be understood by individuals Can be acquired. In addition, because it is based on travel history, you can get more comprehensive information including travel routes and time zones from the information of each store, you can apply it as it is to the travel schedule, or dinner from lunch You can get detailed information such as popularity. In addition, since the database is automatically updated by sending a movement history by an individual, the latest information can always be provided.

  According to the present embodiment described above, the position history recorded at regular time intervals is classified into stay and movement based on speed. The classification result is fed back to the position coordinates in the position history and the stay sections are integrated as appropriate, thereby improving the accuracy of the classification result and clarifying the stay location. At this time, it is possible not only to clarify the place of stay but also to move on the time axis and clarify the section of stay, so that the time flow of human actions can be grasped, and movement history information consisting of various items can be obtained. Can be acquired. In addition, information such as location, route, time zone, and the like can be associated with a movement history, and eventually a personal event, as a tag, which facilitates search and facilitates information management. Further, by converting the position history into the movement history, data can be collected, and the data size can be efficiently reduced.

  Further, by generating a classification pattern with a plurality of speed threshold values, the abstraction level of the place is changed depending on the information to be presented. For example, when the movement history is displayed as a graphic, the information to be displayed can be adaptively changed only by specifying the scale by switching the classification pattern according to the scale of the distance. As a result, the user can efficiently obtain desired information. Furthermore, by combining the movement history and the information distributed on the network, various information related to the place can be presented. In addition, by accumulating a plurality of people's movement histories and integrating information such as staying places and routes, it is possible to always present recommended places and traffic jam information based on new information.

Embodiment 2
In the first embodiment, a movement history such as a staying place and a movement route is generated by analyzing the position history, and the movement history is visualized by displaying it on a map or the like. At the same time, the abstraction level of the staying place was switched seamlessly according to the scale by switching to the classification pattern of the staying section and the moving section extracted at different speed thresholds according to the change in the scale of the displayed map. . Although the present embodiment is common to the first embodiment in that the stay place and route are graphically displayed on the map, the stay places are hierarchized and the map scale and display area are switched depending on the hierarchy to be displayed. The information processing apparatus according to the present embodiment can be realized with the same configuration as described with reference to FIG. 2 in the first embodiment, but the configuration of the behavior information generation unit 16 is different from that in the first embodiment. Hereinafter, illustration and description of portions common to the first embodiment are omitted as appropriate.

  FIG. 17 shows in detail the configuration of the behavior information generating unit 16 in the present embodiment. The behavior information generation unit 16 generates a tree diagram based on the stay location of the movement history, and a screen configuration determination unit 142 determines a map scale and a display area to be displayed based on the stay location to be displayed at the same time. A display image generation unit 144 that generates display image data corresponding to the scale and area to be displayed, a display image storage unit 148 that stores data of the generated display image, and a display that switches an image to be displayed according to an instruction from the user An image switching unit 146 is included.

  The dendrogram generation unit 140 reads the movement history specified by the user via the instruction input reception unit 12 from the movement history storage unit 24, and performs hierarchical clustering based on the stay location, its position, the length of time, and the time. By generating a dendrogram, the place of stay is hierarchized. For this purpose, the maximum number of staying places displayed on one screen, that is, the number of clusters is determined in advance. Then, N clusters that are close to each other are merged to form an upper cluster. Here, the cluster distance is the distance between the centroids of the distribution of the position coordinates included in each cluster among the position coordinates recorded in the position history, or the distance of the contour or the time distance when the distribution is approximated by a figure. is there. By repeating this, the dendrogram is completed. The cluster generated here is displayed as a place of stay when the map is displayed.

  The screen configuration determination unit 142 searches the tree diagram generated by the tree diagram generation unit 140 for all the routes from the top layer to the bottom layer so that the number of clusters to be displayed is N at the same time. By determining the appropriate hierarchy, the necessary screen configuration is determined. At this time, the scale ratio and display area of the image are determined so that N clusters can fit on the screen in any hierarchy of images. The display image generation unit 144 generates display image data including a map of the scale and display area determined by the screen configuration determination unit 142 and a graphic representing a cluster displayed there, and stores the display image data in the display image storage unit 148. . Since the places of stay are gathered at the higher levels, the map is displayed over a wide area.

  The display image switching unit 146 first reads out the initial image data from the display image storage unit 148 and supplies the read data to the output unit 18. For example, an image layer used for the initial image is determined in advance, for example, the highest layer, that is, the widest map is used as the initial image. When the user performs an input for selecting one of the displayed clusters via the instruction input receiving unit 12, an image is displayed that displays the clusters of the layer one layer below the selected cluster at a time. Switch the display.

  FIG. 18 is a diagram for explaining the relationship between a tree diagram and a display image. In the following description, a cluster in which a plurality of stay places are collected is also referred to as a stay place in a broad sense. The dendrogram 150 is generated based on a movement history including seven stay places A, B, C, D, E, F, and G so that three stay places are displayed on one screen. Is. For such a dendrogram, stay places C and D are collectively referred to as stay place A '. The stay place A 'and the stay places A and B are collectively referred to as a stay place A ". The stay place F and the stay place G are collectively referred to as a stay place B".

  In the first image, which is the map of the widest area, as shown by the solid line, three stay places, stay place A ″, stay place E, and stay place B ″ are displayed. When staying place A ″ is selected here, staying places A and B and staying place A ′ are displayed as shown by dotted lines. When staying place A ′ is further selected, it is shown by a dashed line. As shown, the stay places C and D are displayed. When the stay place B ″ is selected, the stay places F and G are displayed as indicated by the two-dot chain line. Therefore, if seven images corresponding to each of the stay locations A to G and four images each having a smaller scale and displaying three stay locations are generated in advance, all necessary scales and display areas are covered. can do.

  Next, the operation of the configuration described so far will be described. FIG. 19 is a flowchart showing a processing procedure for generating and displaying a display image for displaying a movement history. First, when the user inputs an instruction to display a movement history together with a movement history file specification via the instruction input receiving unit 12 (S40), the dendrogram generation unit 140 obtains a movement history file from the movement history storage unit 24. Reading (S42), hierarchical clustering is performed based on the positional relationship of the stay places recorded in the data, and a dendrogram is generated (S44).

  Then, the screen configuration determination unit 142 searches the tree diagram from the uppermost layer to the lower layer, and determines the relationship between necessary images and hierarchies so that a maximum of N staying places are displayed in one image. At the same time, the scale ratio of each image is determined so that all the staying places in each image fall within a predetermined area from the center of the image (S46). Since the lowest level image displays only one place of stay, the distribution of the position coordinates included in the corresponding stay section and recorded in the position history, and the figure approximating it, have an area of a predetermined percentage from the center of the image. The scale ratio of each image is determined so as to fit. The display image generation unit 144 acquires the information, generates image data corresponding to each layer, and stores it in the display image storage unit 148 (S48). At this time, the map data to be used may be downloaded via a network.

  The display image switching unit 146 reads out the uppermost image data to be displayed first from the display image storage unit 148 and displays it (S50). If the image data is generated from the uppermost image data in S48, the display image switching unit 146 starts the display of S50 while the display image generating unit 144 generates the subsequent image data in the background. Can do. When there is no instruction to end the display from the user and an instruction is input to select one of the stay location figures included in the display image (N in S52, Y in S54), the display image switching unit 146 displays the selected stay. The display is switched to the lower layer image of the place (S56, S50).

  The display of the same image is continued until the stay place selection input is made (N in S54, S50). The processing from S50 to S56 is repeated until the display end instruction is input from the user (N in S52, S54, S56, S50). If an instruction to end the display is given, the process ends (Y in S52). When an image other than the uppermost layer is displayed, an instruction to return the image to the upper layer is accepted in S54. For this reason, for example, a tree diagram or a figure that can grasp the hierarchy may be displayed on the image so that an input for selecting the upper layer can be made, or a button (not shown) of the instruction input receiving unit 12 is previously provided. Such a function may be assigned.

  In an application involving map display, when the user can change the scale of the map or move the display area in parallel, the operation for changing the scale and the operation for moving the display area are generally independent. Therefore, when the user wants to see a desired area at a desired scale, it is necessary to adjust both while viewing the screen. Therefore, if the scale of the map is increased, a region different from the intended one may be displayed or a desired region may be lost.

  In addition, especially in an aspect in which map image data is downloaded by connecting to a network, various processes for displaying a new image are required every time such a fine adjustment of the user is performed. It takes time to display a new image. Depending on the processing performance of the device and the state of the network line, this time cannot be overlooked and can be stressful for the user.

  In this embodiment, as described above, paying attention to the stay location of the movement history, and classifying and hierarchizing based on the number and the positional relationship, the display area and the scale in which the stay location is displayed in a well-balanced manner are determined in advance. . When a certain staying place is selected, the lower layer image is displayed so that the display can be directly switched to the scale and display area desired by the user. In addition, since the display image is determined in advance, the image data can be downloaded first and the display image is generated independently of the switching process. Time lag is unlikely to occur between operation and display.

  20 to 22 show examples of images that are actually displayed. In this example, the movement history in one trip is displayed, and the stay / movement section is classified with a speed threshold of 25 km / h. By this classification, it is possible to consider staying when the speed is lower than walking or moving by car when it is faster. As described above, the maximum number of staying places displayed on one screen is set to 3, and the scale of the map is determined so that the staying place displayed on one screen falls within 75% of the area from the center of the map.

  A hierarchized movement history display image 150a in FIG. 20 is an uppermost image, and on the map 152, a double circle graphic 154a representing a staying place and a double line graphic 156 representing a movement route are represented. The display method is the same as that described in the first embodiment. The hierarchical movement history display image 150a further includes a tree diagram display field 158a. The dendrogram display column 158a shows a dendrogram as a result of classifying the stay places by hierarchical clustering. Since each leaf node of the dendrogram corresponds to the minimum unit stay location included in the travel history, information related to the stay such as the date and time of stay at each location is read from the travel history and displayed.

  The hierarchy corresponding to the hierarchized movement history display image 150a in FIG. 20 is clarified by making the line of the tree diagram in the tree diagram display field 158a thick. Alternatively, the line color may be changed. In the figure, the location number indicated in the leaf node is a cluster in which the stay locations of [0], [1], [2], and [3] are collected, the stay location of [6], [4] and [5] ] Are shown as three stay places. The hierarchical movement history display image 150 a further displays a character 160 a that can be moved by the user by the instruction input receiving unit 12. The character is used for selecting a staying place.

  In the state of FIG. 20, the character 160a is at the place where the double circle figure 154a stays. The thickest line among the thick lines in the tree diagram display field 158a indicates which level of the tree diagram the stay place corresponds to. Represented by In this example, it can be seen that a cluster in which [0], [1], [2], and [3] are gathered is currently being selected. Furthermore, a graphic representing the number of clusters in the next lower layer is displayed in the vicinity of the staying place being selected. In the example of FIG. 20, three small double circles 162a are displayed to indicate that the number of lower layer clusters is three. This corresponds to a stay location of [1], a stay location of [3], and a cluster that combines [0] and [2].

  When the user inputs an instruction to determine the selection in this state, an image shown in FIG. 21 is displayed. The map shown by the hierarchized movement history display image 150b in FIG. 21 represents the vicinity of the stay location selected in FIG. 20, and as described above, the lower layer cluster of the selected stay location is 75 of the entire image. % Of the scale so as to fit in the% area. The display method of the hierarchized movement history display image 150b is the same as that of the hierarchized movement history display image 150a of FIG. 20, but the staying place of [1], the staying place of [3], and [0] and [ 2] is displayed. Therefore, the hierarchy corresponding to the dendrogram display column 158b is indicated by a bold line.

  Of the staying places being displayed, the staying place of the double circle figure 154b in which the current character 160b is present is not a cluster of a plurality of staying places, but is a minimum staying place. It is represented by the number of small double circles 162b. Further, in the dendrogram display column 158b, information on the corresponding staying place is indicated by a bold line and underlined. When the user inputs an instruction to decide the selection in this state, an image shown in FIG. 22 is displayed.

  The hierarchized movement history display image 150c in FIG. 22 displays a map in the vicinity of the stay location selected in FIG. Since the image is an image including one stay place of the minimum unit, the map displayed as described above is included in the stay section corresponding to the stay place, and the distribution of the position coordinates recorded in the position history is 75. % Of the scale so as to fit in the% area. The display method is the same as in FIGS. 20 and 21, but the stay location is displayed as a single circle figure 164 because it is the minimum unit stay location. Also, in the dendrogram display column 158c, information on the corresponding staying place is represented by a bold line and underlined.

  In the present embodiment, as described above, when one of the stay places included in the image of the upper hierarchy is selected, a map having a display area and a scale ratio in which the stay places below the selected stay place fall within a predetermined range. Is displayed. At this time, the direction of the viewpoint with respect to the map to be displayed may be diagonal other than directly above, and may be freely set by the user or the system. Therefore, necessary map data can be specifically determined by generating a dendrogram. The display image switching unit 146 may handle the plurality of maps determined as described above independently, and switch and display them. However, the display image switching unit 146 integrates each image as one image and switches the viewpoint. Image switching may be realized.

  FIG. 23 is a diagram for explaining the relationship between the image generated in this case and the viewpoint. In the figure, “Map 4” is the image at the top layer, and “Map 1”, “Map 2”, and “Map 3” are the images at the lower layer. As described above, since the maps with the increased scale ratio of the specific place of “Map 4” are “Map 1”, “Map 2”, and “Map 3”, “Map 1” is included in “Map 4”. There are areas corresponding to “Map 2” and “Map 3”. Therefore, as shown in the figure, the lower-level maps “Map 1”, “Map 2”, and “Map 3” are pasted to the corresponding areas on the upper-level map “Map 4”.

  Since “Map 1”, “Map 2”, and “Map 3” are maps with a high scale ratio, more detailed information than the corresponding area of the map 4 is described. Therefore, a single map with partially detailed portions can be obtained. A viewpoint 170 is set for the map generated in this way. The viewpoint where the entire uppermost image “map 4” displayed as the initial image overlaps the field of view is defined by the x and y coordinates as the image plane and the z coordinate as the height of the viewpoint. Similarly, for “Map 1”, “Map 2”, and “Map 3”, viewpoints that entirely overlap with the field of view are defined. The z coordinate of the viewpoint displaying “Map 1”, “Map 2”, and “Map 3” is smaller than the z coordinate of the viewpoint displaying “Map 4”.

  One image data and the coordinates of the viewpoint corresponding to each map are stored in the display image storage unit 148. The display image switching unit 146 performs image switching as a change in viewpoint with respect to one image. When the movement of the viewpoint is completed, a figure such as a corresponding staying place is displayed in an overlapping manner. A technique generally used in the field of computer graphics or the like can be applied to a method for determining a display image by defining a viewpoint. When the images are switched by moving the viewpoint in this way, for example, when a stay place is selected in the upper image, it can be expressed as zooming up there, and seamless and natural switching is possible.

  According to the present embodiment described above, in a mode in which stay places and travel routes are displayed on a map, the minimum unit stay places are classified by hierarchical clustering to generate a dendrogram. Then, based on the predetermined number of clusters to be displayed in one image and its position, the scale and display area of the map to be displayed are determined, and image data to be displayed is generated in advance. When one of the staying places is selected at the time of displaying an image, the display is switched to an image of a scale suitable for the display including the lower staying places gathered in the staying place. In this way, the user can easily shift the display to a desired scale ratio and a desired display area. Further, since display image data can be determined in advance, it is not necessary to download image data every time the scale ratio or display area is changed, and display with good responsiveness can be performed.

  Furthermore, when the map displayed on the lower layer image is pasted on the map displayed on the upper layer image data is generated, the viewpoint for the one image is moved when switching the image. I will let you. As a result, even when the scale of the display image is changed, seamless image transition is possible with the minimum necessary image data.

Embodiment 3
In Embodiments 1 and 2, the movement history is visualized by deriving the movement history from the position history and displaying the movement history on a map or the like. In the present embodiment, in a slide show that automatically switches and sequentially displays photos selected from a plurality of photos taken during a trip or the like, moving pictures that are displayed with effects or BGM are displayed as movement history information. Use to select. Since the selected photo conforms to the movement history, it can be displayed together with the image display of the movement history. The information processing apparatus according to the present embodiment can be realized with the same configuration as described with reference to FIG. 2 in the first embodiment, but the configuration of the behavior information generation unit 16 is different from those in the first and second embodiments. Hereinafter, illustration and description of portions common to the first and second embodiments are omitted as appropriate.

  FIG. 24 shows the configuration of the behavior information generation unit 16 in the present embodiment in detail. The behavior information generation unit 16 includes an image data acquisition unit 210 that acquires image data of a photograph specified by the user, an image data storage unit 212 that stores image data of a photograph taken by the user, and a movement history specified by the user. A display number determination unit 214 that determines the number of images, an image selection unit 216 that selects the determined number of photos and extracts data, and a display image generation unit 218 that generates display image data including photos and movement history.

  The image data acquisition unit 210 reads, from the image data storage unit 212, a series of photo data specified by the user via the instruction input receiving unit 12. The image data acquisition unit 210 further acquires image data from the information providing server 8 as necessary. The display number determination unit 214 reads from the movement history storage unit 24 the movement history corresponding to the photo data or the position information provided as additional information of the photo specified by the user via the instruction input receiving unit 12. Then, considering the stay section and the travel section on the time axis from the start to the end of the travel history, the number of photographs to be displayed for each stay section and the travel section is determined.

  In determining the number of sheets, the importance of stay / movement, the time ratio of each section, and the number of photographs taken in each section are considered. Therefore, the display number determination unit 214 acquires the shooting time of each photo from the additional information of the photo data from the image data acquisition unit 210. Then, the number of photographs taken in each section is derived from the start time and end time of each section in the movement history and the photographing time of the photograph. A specific method for determining the number of photographs displayed for each section using such parameters will be described later.

  The image selection unit 216 arranges a plurality of photos acquired by the image data acquisition unit 210 in chronological order, and selects a photo to be displayed from photos taken in each section according to the determination of the display number determination unit 214. At this time, the selection is made in consideration of the contents of the photograph. For example, information such as the balance between a portrait photograph and a landscape photograph and the shooting situation added to the photograph data is considered. The display image generation unit 218 acquires the selected photo data from the image selection unit 216, applies a frame or an effect such as cross fade, and sequentially outputs the photo images to the output unit 18. Thereby, on the display screen which is the output unit 18, moving image display configured based on the selected content can be realized.

  Note that the display image generation unit 218 may not only display the selected photos sequentially but also display them together with the movement history as will be exemplified later. In other words, in this embodiment, since a photograph is selected for each section of the movement history, correspondence with the stay location and movement route of the movement history can be taken. Therefore, it is possible to easily realize a mode in which the slide show and the movement history are displayed simultaneously using the correspondence.

  Next, a method in which the display number determination unit 214 determines the number of photographs to be displayed will be described. FIG. 25 is a diagram for explaining a method of determining the number of photos to be displayed for each section on the time axis in the movement history. In this figure, the horizontal direction is the time axis, the nth stay section “stay (n)”, the nth travel section “movement (n)”, the n + 1th stay section “stay (n + 1)”, the n + 1th The stay section and the movement section are alternately arranged as the movement section “movement (n + 1)”,. On the other hand, the image data storage unit 212 collectively stores image data of a plurality of photographed photographs 220 in the order of photographing.

  Therefore, by classifying the photos into each section of the movement history according to the shooting time, it is difficult to judge from the contents of the photos, and the human behavior is associated with the photos. And by selecting the photos to display from each section, even if the total display time is limited, the display does not suddenly shift from one photo of the place of stay to the photo of another place of stay. A slide show that preserves

First, the total display number N_all is determined from the predetermined total display time of the slide show and the display time per sheet. On the other hand, the ratios l_move and l_stay of the number of sheets displayed in the movement section and the stay section are also determined in advance. Here, l_move + l_stay = 1. The ratio T (n) of the time of each section with respect to the total time of the movement history and the ratio S (n) of the number of photographs taken in each section with respect to the total number of photographed pictures are respectively obtained from the movement history and the photograph data. To derive. Using these parameters, the number N_move (n) of photos displayed in the nth moving section and the number N_stay (n) of photos displayed in the nth stay section are determined as follows.
N_move (n) = N_all × l_move × α × T (n) × β × (m + S (n))
N_stay (n) = N_all × l_stay × α × T (n) × β × (m + S (n))

  Here, α and β are reflection coefficients of the parameters T (n) and S (n), respectively, and by changing relatively, either the proportion of time in each section or the number of photographs taken in each section is selected. You can adjust whether to prioritize the number of displayed images. M is a constant for preventing the number of displayed pictures from being zero even when the number S (n) of photographed pictures is zero. Since the displayed number is a natural number, when a fraction is obtained, for example, the displayed number of the section with the largest number of shots is increased by one and the other sections are discarded.

  Next, a method in which the image selection unit 216 selects a photo to be displayed will be described. FIG. 26 is a diagram for explaining a method of selecting a predetermined number of photos from photos taken in a certain section in the movement history. In the figure, the horizontal direction is the time axis, and shows only a certain section, that is, the movement section “movement (n)” or the stay section “stay (n)”. The lower row is an array of photographs taken in this section, and the number of photographs corresponds to S (n) in the above formula. The upper row is a sequence of photos selected from the upper row, and the number of the photos is N_move (n) or N_stay (n) determined by the display number determination unit 214.

  At this time, for example, it is possible to select according to a predetermined rule based on a sequence of photos such as every other photo, or to generate a random number and select randomly, but by selecting without considering the content of the photo, It is possible that only photos that tend to be less desired by the user are selected. In addition, due to the nature of the slide show, it is desirable to select photographs with contents that capture the key points of events such as travel. Therefore, in this embodiment, a photograph showing a person and a photograph other than that are classified in advance so that, for example, a portrait photograph and a landscape photograph are displayed in a well-balanced manner.

For this purpose, face detection is first performed using the face recognition technique on the photograph taken in the lower part of FIG. Then, the photograph in which the face is detected is classified as a person photograph, and the other photograph is classified as a non-person photograph. If it is a photograph taken during a trip or the like, the non-person photograph is often a landscape photograph. On the other hand, the ratio H (0 ≦ H ≦ 1) of the number of person photographs among all displayed photographs is determined in advance. This value may be fixed in the information processing apparatus 10 or may be adjusted by the user. Then, the display number Va (n) of person photographs and the display number Vb (n) of non-person photographs in each section are obtained as follows.
Va (n) = H × (N_move (n) or N_stay (n))
Vb (n) = (1-H) × (N_move (n) or N_stay (n))

  The fraction may be assigned to either one or may be rounded down. The image selection unit 216 selects portraits and non-portraits so that the final number is Va (n) and Vb (n). To do this, the following processing is performed on each of the portrait and non-portrait photos. First, in the sequence of photographed images, the two photos that are closest in time are picked up and compared, and one of them is removed from the selection candidates by the beautiful photo determination. If there are two other photos of the same closeness, they are also compared and excluded from the selection candidates. Pick up the two photos that are closest in time among the remaining photos and remove one of them from the selection candidates.

  Such a process is repeated until a final picture number Va (n) or Vb (n) is left. Since photos that are close in time are likely to be similar in composition, removing selection candidates in this way can prevent duplicate photos of similar composition from being selected in addition to time. Can be selected evenly.

  Here, the beautiful photo determination is to give a superior or inferiority to the contents of a photo with a predetermined index. When using a plurality of indices, for example, by assigning a score according to a predetermined rule for each index, a total score is calculated. Compare. For example, in the case of a person photograph, the result of face recognition for extracting the person photograph is used, and the higher the ratio of the total area of faces facing the front in the screen, the higher the score. In addition, when the number of faces facing the front, that is, the number of people is combined, adjustments such as increasing the number of people who have a large number of people or a large number of people with the same total area of the face, etc. It becomes possible. Further, as many people as possible may appear evenly based on attributes such as race, sex, and age obtained as a result of face recognition.

  In addition to using such a face recognition result, additional information attached to data of each photograph such as an EV (Exposure Value) value, a shutter speed, and an angle of view may be used. For example, a high-scoring photo with a fast shutter speed can make it easy to select a photo that is not blurred, and a non-portrait photo can make it easy to select a landscape photo by making a wide-angle photo a high score. In this way, the rules for scoring a person photograph and a non-person photograph may be different.

  When the number of photographed non-person photographs is less than the number of non-person photographs displayed Vb (n), the image selection unit 216 notifies the image data acquisition unit 210 to that effect, and the image data acquisition unit 210 A photo provided for free or for a fee may be downloaded from the information providing server 8 by a predetermined procedure. Or you may read from the content collection etc. which hold | maintain offline. At this time, the image data acquisition unit 210 refers to the movement history and the position history, and specifies the latitude, longitude, etc. of the position coordinates included in the section lacking the photograph, thereby acquiring a suitable photograph for the section. Is desirable.

  For this reason, it is desirable that the information providing server 8 has a database in which photographs are classified with respect to positions such as latitude and longitude. However, even if detailed settings such as latitude and longitude are not used, a similar effect can be obtained by searching for and obtaining a photo on the network based on a place name or facility name associated with each section of the movement history. Even when the number of photographed person photographs is less than the number of person photographs displayed Va (n), it may be supplemented with photographs obtained from the information providing server 8.

  Next, the operation of the configuration described so far will be described. FIG. 27 shows a processing procedure for performing a slide show with photos accompanied by movement history information. First, when the user designates the movement history and the data of the photograph taken in the movement and inputs an instruction to start a slide show through the instruction input receiving unit 12 (S60), the image data acquisition unit 210 and the display number determination unit 214 The designated photo data and movement history are read out from the image data storage unit 212 and the movement history storage unit 24, respectively (S62). Then, the display number determination unit 214 determines the number of displayed photographs for each stay / movement section included in the movement history as described above (S64).

  Next, the image selection unit 216 acquires the data of the processing target photo from the image data acquisition unit 210, and then performs face recognition processing on all the photos and classifies the photos into portraits and non-portraits (S66). . Then, for each section, the photograph is selected from each category so that the ratio of the number of person photographs to non-person photographs becomes a predetermined value (S68). If the photographed photograph has not reached the number of images displayed for that section (N in S70), the image selection section 216 notifies the image data acquisition section 210 to that effect, so that the image data acquisition section 210 An insufficient number of photos are downloaded from the information providing server 8 based on the position information (S72).

  When the photos to be displayed are ready (Y in S70 or S72), the display image generation unit 218 generates an image for switching the photos in order of shooting time, and outputs them to the output unit 18 to display the slide show (S74). . In the same section, the photographs may be displayed in an order different from the order of the shooting times for production. For example, the order may be changed so that portrait photographs and landscape photographs are alternately displayed.

  Next, an example of a display image generated by the display image generation unit 218 is shown. In this example, the map display of the movement history described in the second embodiment is used, and the selected photos are displayed in the form of a slide show in synchronization with a character that moves on the map following the history. Realize a certain expression. As described above, in the present embodiment, since a photo is selected in correspondence with each section of the movement history, it is possible to easily develop a slide show together with the movement history. However, the present embodiment is not limited to this, and only selected photos may be switched and displayed in a predetermined time. In addition to the map, it may not be a map as long as it represents a surface of the earth such as an illustration or aerial photograph.

  28 to 30 show examples of images displayed in a form in which the movement history and the slide show are displayed in synchronization. Actually, it is expressed by continuous motion, but in the figure, several frames are extracted and shown. The movement history used is the same as that illustrated in FIGS. 20 to 22 of the second embodiment. That is, the photos selected as described above from the photos taken in various places in one or more trips are displayed in a slide show. First, the movement history display image 230 in the upper part of FIG. 28 displays a graphic 236 representing a movement route and a graphic 238 representing a staying place on a map 232 as described in the second embodiment. Although the character 234 is also displayed, unlike the second embodiment, the character 234 moves based on the movement history.

  An existing technique can be used for the animation display of the character 234. The movement history display image 230 in the upper part of FIG. 28 is a display image when the character 234 arrives at the stay location represented by the graphic 238 through the movement route. Considering the time axis of the movement history, the character 234 moves on the map according to the data of the first movement section of the movement history, and reaches the first stay section. Thus, when the character 234 arrives at a certain staying place, the display is switched to a staying period photo display image 240 as shown in the lower part of FIG.

  The stay period photo display image 240 displays a photograph in the main area 244, and shows a map of the stay place that is the current location of the character in the sub area 242 having a smaller area. This map image corresponds to the lowest layer image in the second embodiment. In the main area 244, the photos selected for the corresponding stay section are switched and displayed every predetermined time. This makes it possible to clearly understand the correspondence between the staying place and the photograph.

  When all the selected photos are displayed, the display is switched to a moving period photo display image 250 as shown in the upper part of FIG. The movement period photo display image 250 displays a map representing the movement history in the main area 252 and displays a photo in the sub area 254. A character 256 is also displayed on the map of the main area 252. On the time axis of the movement history, the character 256 moves on the map according to the data of the movement section that has arrived after leaving the previous stay section. In the sub-region 254, in synchronization with the movement of the character, the photograph selected for the moving section is switched and displayed at predetermined time intervals. The speed at which the character 256 moves is determined from the distance of the moving section and the display time per picture so that the movement of each moving section and the display of the pictures are completed at the same time.

  The map displayed in the main area 252 in the moving period photograph display image 250 is the lowest layer image in the second embodiment immediately after the character 256 leaves the place of stay. When the character 256 moves away from the staying place according to the movement history data, the display is switched to the map on the upper layer in the second embodiment. In order to perform such control, a reduction direction image switching boundary 258 when the character 256 moves in a direction away from the place of stay is determined in advance.

  Although the reduction direction image switching boundary 258 is not actually displayed, as shown in the figure, by setting the character 256 at a position before it deviates from the screen, the scale and display area of the image are appropriately maintained while maintaining continuity. You can switch to The image switched in this way is a moving period photograph display image 260 in the lower part of FIG. At this time as well, a map representing the movement history is displayed in the main area 262, and a photograph is displayed in the sub area 264.

  The map displayed in the main area is a map higher than the map displayed in the upper-stage moving period photo display image 250. A dendrogram 269 similar to that shown in Embodiment 2 may be further displayed so that such a hierarchy can be understood. Similar to the moving period photo display image 250, the character 266 moves on the moving path with the passage of time, and accordingly, the photo of the sub-region 264 is also switched and displayed. By doing so, the scenery of the route along which the character 266 is moving is displayed with a sense of reality. Even if there are not enough photographs taken, it is possible to express a sense of incongruity by including a display of a landscape photograph of the place acquired from the information providing server 8.

  When the character 266 approaches the next staying place, the display is switched to the lowest map corresponding to the staying place. In order to perform such control, an enlargement direction image switching boundary 268 when the character 266 moves in the direction approaching the staying place is determined in advance. The enlargement direction image switching boundary 268 is not actually displayed. When the character 266 exceeds the boundary, by switching the display to the lower layer image described in the second embodiment, an image with an appropriate scale ratio and display area can be easily displayed while maintaining continuity.

  The image thus switched is the moving period photo display image 270 in the upper part of FIG. Also at this time, a map representing the movement history is displayed in the main area 272 and a photograph is displayed in the sub area 274. When the character 276 enters the staying place in this image, one movement section in the movement history ends. At this time, all the photos selected for the movement section are displayed. Then, the stay period photo display screen 280 in the lower part of FIG. 30 is displayed. Similarly to the stay period photo display image 240 in the lower part of FIG. 28, the stay period photo display screen 280 also displays a photo in the main area 282 and a map of the stay place in the sub area 284. In the main area 282, the photos selected for the corresponding stay section are switched and displayed every predetermined time.

  By repeating such processing for all moving sections and staying sections of the movement history, a slide show synchronized with the movement of the character indicating the location on the map can be realized. As a result, it is possible to display the photos in a well-balanced manner in accordance with individual actions such as staying and moving of an event having a time width like a trip.

  However, in events such as travel, the frequency of taking photos on the return trip can often be reduced. In such a case, it is conceivable that the displayed content becomes poor as the end of the slide show is approached. Therefore, when the character has finished moving the entire process, a slide show may be further displayed using a predetermined number of photos so as to summarize all the processes. You may select the photograph displayed at this time separately from the photograph displayed with a character in the main part.

  However, the selection algorithm may be the same. For example, the overall picture can be selected in a well-balanced manner only by setting the total number of displayed sheets N_all small. On the other hand, using the face detection results, the photos are classified as follows: photos with no face, photos with one face, photos with two faces, etc. You may make it select on the conditions suitable for summarization, such as selecting the predetermined number of photograph from classification | category. The selection condition may be adjusted by the user. For example, when classifying by the number of faces as described above, the maximum number of faces of photos to be selected is set as the number of family members, so that photos that contain many people other than family members are excluded from the summary. A coherent slide show can be realized.

  In all the above examples, a still image is a target of processing. However, the present invention can be applied to data having a length of time such as voice and moving images. In this case, the number of photos to be displayed in each section is replaced with the length of the moving image or sound. Then, data corresponding to the length determined for each section is extracted from the first or middle part of the data. By outputting it together with the character moving on the map, it is possible to express voice and moving images corresponding to each stay and movement. You may combine a photograph, a sound, and a moving image, such as switching a photograph and a moving image according to a section, or adding a sound to a slide show. In addition to photographs, computer graphics and hand-drawn pictures taken with a scanner, text data such as poems and notes, etc. may be used.

  According to the present embodiment described above, a part of the photos taken at the event in which the position history is recorded is displayed as a slide show. At this time, the number of images to be displayed is determined for each section based on the time ratio of the moving section and the stay section in the travel history derived from the position history, the number of photographs taken in each section, and the ratio of the photographs to be displayed in the moving section and the stay section. , Select a photo. As a result, it is possible to select a photo in accordance with the actual situation of each act of staying and moving.

  In addition, face selection technology is introduced to select photos, and it is not only classified as a person photo or a non-person photo, but also the area ratio and the number of people in the face area are derived and used as criteria for selection. . In addition, by setting the ratio of the number of portraits and non-portraits in advance, a variety of photos can be selected. As a criterion for selecting a photo, additional information of the photo is also used, and evaluation from various angles improves the prediction accuracy of the photo desirable for display. For these reasons, even if the total display time is limited, there is little bias or omission, and it is possible to display a slide show that accurately reflects each situation in an event such as a trip.

  Moreover, since the photo selection is performed based on the movement history, it is easy to associate with the information of the movement history. By utilizing this fact, it is possible to realize an expression method in which the movement of the character moving on the map according to the movement history and the photograph to be displayed are synchronized. By simultaneously controlling the movement of the display area of the map and the switching of the displayed photograph at an appropriate timing in this way, a new expression with a sense of reality that does not fit in a simple slide show can be realized.

  The present invention has been described based on the embodiments. Those skilled in the art will understand that the above-described embodiment is an exemplification, and that various modifications can be made to combinations of the respective constituent elements and processing processes, and such modifications are also within the scope of the present invention. is there.

  For example, in the above-described embodiment, the position history is data in which position coordinates are recorded at a predetermined time interval. However, the time interval may not be constant as long as it is information in which time and position are associated with each other. For example, even when using various information such as a camera, a video camera, a voice recorder, a PDA (Personal Digital Assistants), etc., when recording various data, information on the recording time and recording position is also used. Can be obtained.

  In the third embodiment, as the time division for selecting the content, the time is divided into the stay section and the movement section based on the speed obtained from the time change of the position coordinates. However, the data to be used is not limited as long as the time can be divided. For example, content is acquired while holding a portable device equipped with a pulse meter, and based on the time change of the pulse, the time is divided by the time when the pulse exceeds a predetermined threshold, the time below the threshold, etc. But you can.

  8 Information Providing Server, 10 Information Processing Device, 12 Instruction Input Accepting Unit, 14 Location History Analyzing Unit, 16 Action Information Generating Unit, 18 Output Unit, 20 Location History Storage Unit, 22 Analysis Condition Storage Unit, 24 Movement History Storage Unit, 26 position history acquisition unit, 28 speed history calculation unit, 30 classification unit, 32 integration unit, 34 information formation unit, 102 database generation unit, 104 information provision unit, 106 individual information acquisition unit, 108 information integration unit, 110 inquiry acquisition unit 112 search unit 114 result transmission unit 130 display image generation unit 131 scale change instruction reception unit 132 classification pattern switching unit 133 search key reception unit 134 search unit 135 movement history graphic storage unit 136 additional information storage 137 scale / classification correspondence storage unit, 140 dendritic Figure generation unit, 142 Screen configuration determination unit, 144 Display image generation unit, 146 Display image switching unit, 148 Display image storage unit, 210 Image data acquisition unit, 212 Image data storage unit, 214 Display number determination unit, 216 Image selection unit 218 Display image generation unit.

Claims (7)

The time change of the parameter representing the speed is obtained from the position history recorded by associating the position coordinates of the object with the time, and the period during which the speed below the predetermined speed threshold continues for the predetermined duration threshold or longer is maintained. By classifying the time axis into a stay section and a movement section by extracting as a period,
A staying place identifying unit that identifies staying places corresponding to a plurality of staying sections classified by the classification unit from a distribution of the position coordinates in the staying section;
The travel history information formed by connecting the stay location specified by the stay location specifying unit and the travel route in the travel section in chronological order is generated and stored in the travel history storage unit, and the travel history information is stored in the travel history. A movement history information forming unit that associates the date of recording the location history that is the source of information and stores data of a photograph taken on each day as additional information in the movement history storage unit ;
It displays the additional information to the display device, among the pre-SL movement history plurality of movement history information stored in the storage unit, the movement history information associated with the time period specified by the user on the basis of the additional information displayed A search unit that accepts as a search key and searches for travel history information that includes a stay location within a range of a specified location ;
A behavior information generation unit that visualizes movement history information extracted as a search result by the search unit in a predetermined format;
An information processing apparatus comprising:   The behavior information generation unit distinguishes a stay location and a travel route of the travel history information extracted by the search unit, further distinguishes a plurality of extracted travel history information, and displays them on a diagram representing the ground surface. The information processing apparatus according to claim 1. Said action information generating unit, an information processing apparatus according to the number of stays location included in the specified range, to claim 1 or 2, characterized in that to display per movement history information in which the retrieval unit is extracted.   When the stay location included in the plurality of travel history information extracted by the search unit includes the same stay location or the same travel route, the behavior information generation unit represents the stay location or travel route as a single figure. The information processing apparatus according to claim 1, wherein the information processing apparatuses are integrated and displayed on a map. In the information processing apparatus, a speed history calculation unit reads a position history in which the position coordinates of the object are recorded in association with time from the memory, and obtains a time change of a parameter representing the speed;
The classifying unit extracts a period during which the speed below the predetermined speed threshold in the change in the parameter representing the speed continues for a predetermined duration threshold or more as the stay period, thereby moving the time axis to the stay section. A step of classifying into sections, and
The stay location identifying unit identifies a stay location corresponding to a plurality of classified stay sections from the distribution of the position coordinates in the stay section; and
The movement history information forming unit generates movement history information obtained by connecting the identified stay location and the movement route in the movement section in time series order, stores the movement history information in the storage device, and stores the movement history information in the movement history information. Associating with the date when the position history that was the basis of the recording was recorded, and storing the data of the photograph taken on each date as additional information in a storage device ;
Search unit displays the additional information on the display device, before Symbol of the plurality of mobile history information stored in the storage device, movement history associated with the period specified by the user on the basis of the additional information displayed From the information , searching for travel history information that includes the place of stay within the range of the specified place , accepted as a search key ;
A step in which the behavior information generation unit visualizes movement history information extracted as a search result in a predetermined format and outputs it to an output device;
An information processing method comprising: A position history in which the position coordinates of the object are recorded in association with the time are read from the memory, and a function for obtaining a time change of a parameter representing the speed;
The time axis is divided into a stay section and a movement section by extracting, as a stay period, a period in which a speed that is lower than a predetermined speed threshold in the change in the parameter representing the speed continues for a predetermined duration threshold value or more. The ability to classify,
A function for identifying stay locations corresponding to the plurality of classified stay sections from the distribution of the position coordinates in the stay sections;
The travel history information formed by connecting the specified stay location and the travel route in the travel section in time series order is generated and stored in a storage device, and the travel history information is the origin of the travel history information. A function for associating the history with the date of recording and storing the data of the photograph taken on each date as additional information in the storage device ,
Displays the additional information to the display device, among the pre-Symbol plurality of movement history information stored in the storage device, the movement history information associated with the time period specified by the user on the basis of the additional information displayed, the search A function that searches for travel history information that includes a stay location within the specified location range that was accepted as a key ,
A function of visualizing movement history information extracted as a search result in a predetermined format and outputting it to an output device;
A computer program for causing a computer to realize the above. A position history in which the position coordinates of the object are recorded in association with the time are read from the memory, and a function for obtaining a time change of a parameter representing the speed;
The function of classifying the time axis into a stay section and a movement section by extracting a period during which the speed below the predetermined speed threshold in the speed change continues for a predetermined duration threshold or more as a stay period When,
A function for identifying stay locations corresponding to the plurality of classified stay sections from the distribution of the position coordinates in the stay sections;
The travel history information formed by connecting the specified stay location and the travel route in the travel section in time series order is generated and stored in a storage device, and the travel history information is the origin of the travel history information. A function for associating the history with the date of recording and storing the data of the photograph taken on each date as additional information in the storage device ,
Displays the additional information to the display device, among the pre-Symbol plurality of movement history information stored in the storage device, the movement history information associated with the time period specified by the user on the basis of the additional information displayed, the search A function that searches for travel history information that includes a stay location within the specified location range that was accepted as a key ,
A function of visualizing movement history information extracted as a search result in a predetermined format and outputting it to an output device;
A computer-readable recording medium having recorded thereon a computer program characterized in that the computer is realized.

Images