JP6487545B2 - Recognition calculation device, recognition calculation method, and recognition calculation program - Google Patents

Description

  The present invention relates to a recognition degree calculation device, a recognition degree calculation method, and a recognition degree calculation program.

  In recent years, urban development and architectural planning have been carried out based on various viewpoints. For example, Patent Literature 1 discloses that a person's living environment is evaluated and reflected in a city plan or the like.

JP-A-8-161399

  By the way, in conventional urban development and architectural plans, designers design cities and buildings based on subjectivity, and designs that fully reflect the intelligibility to users of cities and buildings are not performed. There was a problem. In other words, since the intelligibility for users of cities and buildings has not been indexed in the past, the designer cannot recognize what kind of design is easy for the user to understand. There has been a problem that the design in consideration of the flow line, the visibility, and the user's perception of the city and the building cannot be sufficiently performed. For example, because the intelligibility of the user's city or building is not indexed, the design proposal cannot be evaluated from the viewpoint of intelligibility when designing a city or public building, and the plan presented by the designer There was a problem that it was difficult to agree.

  Therefore, the present invention has been made in view of these points, and provides a degree-of-recognition calculation apparatus, a degree-of-recognition calculation method, and a degree-of-recognition calculation program capable of indexing ease of understanding for a user of a city or a building. The purpose is to do.

  The degree-of-recognition calculation apparatus according to the first aspect of the present invention includes an acquisition unit that acquires a moving image indicating a spatial route of a city or a building, a movement pattern of a user's line of sight corresponding to the moving image, and the moving image. An element detection unit that detects an element that is estimated to be recognized by the user in the city or the building based on the movement pattern of the line of sight; and the detected element in the city or the building Creation that creates a mental model showing a map that the user envisions for the city or the building based on the position specifying unit that specifies the position, the detected element, and the position of the specified element Part, the mental model, and the reference information indicating the actual element corresponding to the spatial path and the position of the element, for the user of the city or the building It shows the ease borrow, and a calculation unit for calculating a visibility for the city or the building of the user.

The calculation unit may calculate the degree of recognition based on a degree of coincidence between the element and the position of the element included in the mental model and the element and the position of the element included in the reference information. .
The calculation unit may calculate a complexity indicating the complexity of the city or the building based on the number of the elements detected by the element detection unit.
The element detection unit may detect the element based on a composition corresponding to an image included in the moving image and a movement pattern of the line of sight.

The position specifying unit may specify the position of the element based on a shooting position of the image associated with an image included in the moving image.
The position specifying unit may specify a position corresponding to an image in which the element is detected as a position of the element based on a plurality of continuous images included in the moving image.
The acquisition unit may acquire a computer graphic moving image indicating the city or a building as the moving image.
The acquisition unit may acquire a moving image taken by a wearable terminal attached to the user as the moving image.

The element detection unit may identify a composition pattern of an image included in a moving image photographed by the wearable terminal, and detect the element based on the composition pattern of the image.
The acquisition unit may acquire a movement pattern of the user's line of sight detected by a line-of-sight detection device that detects the line of sight of the user.

  The recognition degree calculation apparatus according to the second aspect of the present invention includes an acquisition unit that acquires a moving image that is captured by a wearable terminal attached to a user and shows a spatial route of a city or a building, and a plurality of images included in the moving image. An element detection unit that detects an element that is estimated to be recognized by the user in the city or the building based on a composition corresponding to each of the above, and a position of the detected element in the city or the building. A position identifying unit for identifying, a creating unit for creating a mental model indicating a map that the user envisions for the city or the building, based on the detected element and the position of the identified element; Based on the mental model and the reference information indicating the actual element and the position of the element in the area corresponding to the spatial path, the city or the building Shows the easiness to understand the relative said user, and a calculation unit for calculating a visibility for the city or the building of the user.

  The recognition calculation method according to the third aspect of the present invention includes a step of obtaining a moving image showing a spatial route of a city or a building and a movement pattern of a user's line of sight with respect to the moving image, which is executed by a computer. Based on the moving image and the movement pattern of the line of sight, detecting an element estimated to be recognized by the user in the city or the building, and the city or the building of the detected element Detecting a position in the map, creating a mental model showing a map that the user envisions for the city or the building based on the detected element and the position of the element, and the mental Based on the model and the reference information indicating the actual element corresponding to the spatial path and the position of the element, the city or the building It shows the easy to understand for the serial user, and a step of calculating the awareness of the city or the building of the user.

  The recognition degree calculation program according to the fourth aspect of the present invention includes a computer that acquires a moving image showing a spatial route of a city or a building and a movement pattern of a user's line of sight corresponding to the moving image, An element detection unit that detects an element that is estimated to be recognized by the user in the city or the building based on the moving image and the movement pattern of the line of sight, the city or the building of the detected element A position specifying unit that specifies a position in the image, a creation that creates a mental model indicating a map that the user envisions for the city or the building based on the detected element and the position of the specified element Part of the city or the building based on the mental model and the reference information indicating the actual element corresponding to the spatial path and the position of the element. It shows the easy to understand for the serial user, calculation unit for calculating a visibility for the city or the building of the user, to function as a.

  According to the present invention, it is possible to index the ease of understanding for a user of a city or a building.

It is a figure which shows the outline | summary of the recognition degree calculation apparatus which concerns on 1st Embodiment. It is a figure which shows the structure of the recognition degree calculation apparatus and wearable terminal which concern on 1st Embodiment. It is a flowchart which shows the flow of the process in which the recognition degree calculation apparatus which concerns on 1st Embodiment calculates a recognition degree. It is a figure which shows the user's eyes | visual_axis in one scene included in the moving image which shows a spatial path | route. It is a figure which shows the example by which the line of the perspective projection was specified with respect to the image contained in the moving image which shows a spatial path | route. It is a figure which shows an example of element specific information. It is a figure which shows another example of element specific information. It is a figure which shows the example which pinpoints the position of an element based on a some image. It is a figure which shows an example of a mental model. It is a figure which shows another example of a mental model. It is a figure which shows the example of creation of a three-dimensional mental model. It is a figure which shows an example of an actual map. It is a figure which shows the example which simplifies a mental model. It is a figure which shows the mental model which reflected the element which the some user detected on the actual map.

<First Embodiment>
Hereinafter, a first embodiment of the present invention will be described.
The present inventor considers that a user perceives an image of a city or a building by the user recognizing an element constituting the city or a building, and the element affects the ease of understanding for the user of the city or the building. Thought to give. And this inventor presumes that a user recognizes an element which constitutes a city and a building based on a geometric composition, a coordinate axis, and a figure which are recognized when a user visually recognizes a city and a building. It has been found that the calculation device 1 calculates the degree of recognition of the user's city or building based on the elements estimated to be recognized by the user.

  FIG. 1 is a diagram illustrating an overview of a recognition degree calculation device 1 according to the first embodiment. The degree-of-recognition calculation apparatus 1 is a computer that calculates the degree of recognition that indexes the user's intelligibility for cities and buildings. In the following description, “city and building” are referred to as “city”.

  Specifically, the recognition degree calculation device 1 acquires a moving image showing a spatial route such as a city and a movement pattern of the user's line of sight corresponding to the moving image from the wearable terminal 2 attached to the user (see FIG. (1)).

  The degree-of-recognition calculation apparatus 1 detects an element that is estimated to be recognized by the user in a city or the like based on the acquired moving image and the movement pattern of the line of sight and the position of the element ((2 in FIG. 1). )). Here, the elements include “path”, “edge”, “district”, “node”, and “landmark”. Details of these elements will be described later.

  Based on the detected element and the position of the element, the recognition degree calculation device 1 creates a mental model that shows a map that the user envisions for a city or the like ((3) in FIG. 1). By calculating the degree of coincidence with the actual map (reference information) indicating the actual element and the position of the element corresponding to the spatial route of the city or the like, the user's recognition degree for the city or the like is calculated ((( 4)). Here, the mental model is composed of at least one of a symbol, a picture, a character, and the like representing the content thought by the user. The mental model is, for example, a mental map that is an image configured based on the user's experience, knowledge and experience with respect to a city or the like.

By doing in this way, the designer of a city etc. can evaluate whether a city etc. are easy to understand for a user based on the recognition degree which the recognition degree calculation apparatus 1 computed. As a result, the designer can apply or apply a structure of a city or the like that is easily understood by the user to the design of a new city or the like. In addition, the user can contribute to evaluation and improvement of easy understanding of cities and the like.
Then, the structure of the recognition degree calculation apparatus 1 and the wearable terminal 2 is demonstrated.

[Configuration of Wearable Terminal 2]
FIG. 2 is a diagram illustrating configurations of the recognition degree calculation device 1 and the wearable terminal 2 according to the first embodiment.
First, the configuration of the wearable terminal 2 will be described. The wearable terminal 2 is, for example, a computer that is worn on the user's head. The wearable terminal 2 is a line-of-sight detection device (eye tracking tool) that captures a landscape and records a viewpoint in the landscape. The user wears the wearable terminal 2 and passes through a spatial route that is a continuous space in a city or the like. The wearable terminal 2 captures a spatial route of a city or the like to generate a moving image and generates user's line-of-sight data. Here, it is assumed that the spatial route is a route designated in advance by a designer or the like who investigates ease of understanding of a city or the like. For example, as information indicating the spatial route, map information such as a guide board in front of a station, map information and route information displayed on a terminal such as a smartphone, route information heard from an acquaintance (for example, oral) For example). The spatial route is a route designated in advance, but is not limited to this, and the user may pass through an arbitrary spatial route. Further, as information indicating the spatial route designated to the user, the recognition degree calculation device 1 can obtain the mental model (mental map) of the city or the like generated for the user in the past or the user visually recognizes the map of the city or the like. A mental model (mental map) may be used.

The wearable terminal 2 includes a storage unit 21 and a control unit 22.
The storage unit 21 is, for example, a ROM (Read Only Memory), a RAM (Random Access Memory), a hard disk, an external storage medium (for example, an SD card (registered trademark)) attached to the wearable terminal 2, an external storage device, or the like. Consists of.

The control part 22 is comprised by CPU, for example. The control unit 22 includes a position information acquisition unit 221, an imaging unit 222, a line-of-sight detection unit 223, and a transmission unit 224.
The position information acquisition unit 221 acquires position information indicating the position (terminal position) of the wearable terminal 2 based on GPS signals received from a plurality of GPS (Global Positioning System) satellites. The position information acquisition unit 221 may calculate position information based on GPS signals received from a plurality of GPS satellites.
The wearable terminal 2 may include, for example, an acceleration sensor (not shown) that detects acceleration and an orientation sensor (not shown) that detects an orientation corresponding to the front direction of the wearable terminal 2. Then, the position information acquisition unit 221 may calculate position information indicating a relative position with respect to a reference position (for example, a position where photographing is started) based on the detected acceleration and direction.

  The photographing unit 222 generates a moving image by photographing a landscape in a direction visually recognized by a user wearing the wearable terminal 2. Here, the generated moving image includes a plurality of images, information indicating a reproduction position (elapsed time from the start of shooting) corresponding to the images, and information indicating a time when shooting of the moving image is started. It may be. The photographing unit 222 stores the generated moving image in the storage unit 21.

  The line-of-sight detection unit 223 detects the coordinates (line-of-sight position) of the line of sight in the moving image as the line of sight of the user in the direction in which the user is viewing. The line-of-sight detection unit 223 acquires direction information indicating the direction in which the wearable terminal 2 is facing, that is, the user's viewing direction. The line-of-sight detection unit 223 generates line-of-sight data in which the time when the line of sight is detected, the line-of-sight position, the viewing direction, and the terminal position acquired by the position information acquisition unit 221 when the line of sight is detected is generated, and the storage unit 21 Remember me.

  Here, the moving image and the line-of-sight data are generated in parallel, and it is assumed that there is line-of-sight data corresponding to each of a plurality of images included in the moving image. It should be noted that a moving image in which a pointer indicating the user's line of sight is combined may be generated by the cooperation of the photographing unit 222 and the line-of-sight detection unit 223. The line-of-sight detection unit 223 may detect the locus of the line-of-sight movement pattern. In addition, the control unit 22 may analyze each of a plurality of images included in the moving image photographed by the photographing unit 222 and detect a composition indicated by the image.

  In the present embodiment, the line-of-sight data includes information indicating the terminal position acquired by the position information acquisition unit 221. However, the present invention is not limited to this. For example, without including the terminal position in the line-of-sight data, the control unit 22 may generate information that associates the reproduction position in the moving image and the terminal position of each of the images included in the moving image.

  The transmission unit 224 performs wireless communication with an external device such as the recognition degree calculation device 1. The transmission unit 224 receives the moving image and the line-of-sight data acquisition request received from the recognition degree calculation device 1, and the moving image captured by the imaging unit 222 and the line-of-sight detection unit 223 are stored in the storage unit 21. The generated line-of-sight data is transmitted to the recognition degree calculation device 1.

[Configuration of recognition level calculation device 1]
Then, the structure of the recognition degree calculation apparatus 1 is demonstrated. The recognition level calculation device 1 includes an input unit 11, a display unit 12, a storage unit 13, and a control unit 14.
The input unit 11 is configured by, for example, a keyboard and a mouse. The input unit 11 receives an operation input from an operator of the recognition degree calculation device 1. In addition, when the recognition degree calculation apparatus 1 can communicate with the wearable terminal 2, the input unit 11 of the recognition degree calculation apparatus 1 includes a button, a sensor terminal, and the like, and the wearable terminal includes the button, the sensor terminal, and the like. Line-of-sight input to 2 and gesture input by the user's hand or the like within the range (within the user's field of view) that can be photographed by wearable terminal 2 may be accepted.
The display unit 12 includes, for example, a liquid crystal display or an organic EL (Electro-Luminescence) display. The display unit 12 displays, for example, the moving image visually recognized by the user, the user's mental model, the user's degree of recognition with respect to the city, and the like under the control of the control unit 14.

  The storage unit 13 includes, for example, a ROM, a RAM, a hard disk, and an external storage device connected to the recognition degree calculation device 1. Here, the external storage device may be directly connected to the degree-of-recognition calculation device 1 or may be communicably connected via a communication network (not shown).

  The storage unit 13 stores a recognition degree calculation program that causes the computer to function as an acquisition unit 141, an element detection unit 142, a position specifying unit 143, a creation unit 144, and a calculation unit 145, which will be described later. In addition, the storage unit 13 stores moving images and line-of-sight data acquired from the wearable terminal 2.

The control part 14 is comprised by CPU, for example. The control unit 14 includes an acquisition unit 141, an element detection unit 142, a position specification unit 143, a creation unit 144, and a calculation unit 145. Hereinafter, functions provided in the control unit 14 will be described with reference to flowcharts.
FIG. 3 is a flowchart showing a flow of processing in which the recognition degree calculation device 1 according to the first embodiment calculates the recognition degree.

[Acquisition of motion and eye movement patterns]
First, the acquisition part 141 acquires the moving image which shows the space route of a city or a building, and the motion pattern of the user's eyes | visual_axis corresponding to the said moving image (S1). Specifically, the acquisition unit 141 transmits a moving image and line-of-sight data acquisition request to the wearable terminal 2 and receives the moving image and line-of-sight data from the wearable terminal 2.

  Subsequently, the acquisition unit 141 analyzes the movement of the line of sight of the user indicated by the acquired line-of-sight data, and identifies the pattern of movement of the line of sight indicated by the line-of-sight data. FIG. 4 is a diagram illustrating a user's line of sight in one landscape included in the moving image illustrating the spatial route according to the first embodiment. In FIG. 4, a left line of sight VL indicating the line of sight of the left eye, a right line of sight VR indicating the line of sight of the right eye, and an intermediate point VI indicating an intermediate point between the left line of sight VL and the right line of sight VR are displayed. For example, the acquisition unit 141 analyzes the trajectory of the intermediate point VI to look around, specify confirmation, and gaze as the movement pattern of the user's line of sight. In the example shown in FIG. 4, the left line of sight VL, the right line of sight VR, and the intermediate point VI are displayed. However, the present invention is not limited to this, and only one line of sight may be displayed.

  “Turning” is, for example, visual recognition of the same visual object for 1/5 second or less. “Turning” is a visual recognition that the user mainly locates himself / herself in space. In addition, the “look around” is a state in which, for example, the user does not grasp what an element is just by looking at a range including an arbitrary element. The acquisition part 141 determines with the user having looked around the scenery currently displayed on the said position, when the visual recognition time of the same visual recognition object is 1/5 second or less.

“Confirmation” is a visual recognition of the same visual target longer than 1/5 second and shorter than 1 second. “Confirmation” is visual recognition in which the user localizes himself / herself in space, or visual recognition in which the user localizes the visual recognition target. “Confirmation” is a state in which the user knows that an arbitrary element exists but does not know what the element is.
“Gaze” is visual recognition of the same visual target for 1 second or longer. “Gaze” is visual recognition in which the user confirms the contents of the visual recognition target. “Gaze” means that the user has a figure, form, character, etc. (for example, shape of windows, doors, etc., pattern of finishing material, form of building, form of plant, form of furniture, form of car, form of person, sign Information, map, computer screen, etc.) can be read.

[Detection of elements recognized by the user]
Subsequently, the element detection unit 142 detects an element that is estimated to be recognized by the user in a city or the like based on the moving image acquired by the acquisition unit 141 and the line-of-sight movement pattern (S2).

Here, the elements include paths, edges, districts, nodes, and landmarks proposed by Kevin Lynch. Paths, edges, districts, nodes, and landmarks are defined as follows.
A “pass” is a path that a user may take on a daily or occasional basis or that a user may pass.
An “edge” is a linear element different from a path among linear elements, for example, a boundary between two elements such as a coastline, a railroad track, an edge of a development site, and a wall.
“District” refers to a portion of a city having a medium to large size, and is an element having a two-dimensional extent. The user enters the inside of the “district”. Normally, the “district” is recognized from the inside, but if it is visible from the outside, it is also referred to from the outside.
A “node” is a major point within a city. The user can enter the “node”, go to the “node”, and start from the “node”.
“Landmarks” are things that the user visually recognizes from the outside, and are simply defined things such as buildings, signboards, shops, and mountains. A “landmark” is a thing that the user can visually recognize from the outside without entering the inside.

  Specifically, first, the element detection unit 142 analyzes each of a plurality of images included in the moving image, and identifies a one-point perspective composition or a two-point perspective composition as a composition corresponding to the image. . More specifically, for example, the element detection unit 142 binarizes each pixel value of a plurality of images included in the moving image by brightness, and specifies a binary boundary line as a perspective projection line. Then, the element detection unit 142 specifies a composition corresponding to the image based on a combination of a plurality of perspective projection lines specified in the image. FIG. 5 is a diagram illustrating an example in which a perspective projection line is specified for an image according to the first embodiment. In FIG. 5, it can be confirmed that a one-point perspective composition is specified for the image.

  In addition, although the element detection part 142 specified the line | wire of the perspective projection which can be drawn on an image by binarizing a pixel value, it is not restricted to this. The element detection unit 142 analyzes figures and coordinate axes using a perspective structure, multi-values of pixels based on saturation, extraction of lines from an image using a Hough transform algorithm, and before and after the time when the image was taken In addition, it is possible to specify a plurality of pixels based on a change in the pixel value of the captured image, or to combine them, and to specify a line of the perspective projection that can be drawn. Furthermore, the element detection unit 142 stores a plurality of images corresponding to the composition of the perspective projection in the storage unit 13 in advance, and the plurality of images included in the moving image acquired by the acquisition unit 141 are stored in the storage unit 13. The composition of the perspective projection method may be specified by collating with the image that is present.

  Subsequently, the element detection unit 142 detects an element estimated to be recognized by the user based on the identified composition and the movement pattern of the user's line of sight. Specifically, the element detection unit 142 is assumed to be recognized by the user by referring to the element specifying information that associates the element, the composition pattern, and the line-of-sight movement pattern illustrated in FIG. 6A. Detect elements. For example, when the composition of the image included in the moving image is a one-point perspective composition, and the line-of-sight movement pattern corresponding to the image is gaze, the element detection unit 142 uses “ "Path" is detected.

  The element detection unit 142 may specify an element based on the phase. Here, the phase is a geometric area constructed by various physical events perceived by the user visually, and a concept such as the inside or the outside can be derived. For example, when the user can be conscious of being inside with an emotion such as being consciously wrapped in an arbitrary object (element) or conscious of an arbitrary object (element) from the outside You may be conscious of being outside with emotion. On the other hand, the element detection part 142 detects a phase, for example based on the composition of the image contained in a moving image, and the coordinate axis in the said composition. The element detection unit 142 may further detect the phase based on the movement pattern of the line of sight and the position of the object visually recognized by the user in the image. For example, when the line-of-sight movement pattern is looking around and the look angle is narrow, the user tends to have a phase inside the looked object, and when the look angle is wide, the user However, there is a tendency that a phase exists outside the object that is looked around. Further, when the line-of-sight movement pattern is a plurality of confirmations on the object, the user tends to have a phase inside the object. Further, when the line-of-sight movement pattern is gaze, the phase tends to exist outside the object that the user gazes at.

FIG. 6B is a diagram illustrating another example of the element specifying information. In the element specifying information shown in FIG. 6B, an element, a composition pattern, a line-of-sight movement pattern, and a phase are associated with each other. Even if a plurality of elements are associated with the combination of the line-of-sight movement pattern and the composition pattern, the element detection unit 142 can specify one element based on the phase. it can.

  Further, the element detection unit 142 may identify a composition pattern of a plurality of images included in a moving image captured by the wearable terminal 2 and detect an element based on the composition pattern of the image. For example, a composition pattern of a plurality of images and information indicating an element may be associated with each other and stored in the storage unit 13, and the element associated with the composition pattern identified by the element detection unit 142 may be identified. . The element detection unit 142 detects an area where a geometric coordinate axis can be formed from the composition pattern, and detects an element of the city based on the area, a visual movement pattern, and a phase. Also good.

[Locating detected elements]
Subsequently, the position specifying unit 143 specifies the position of the element detected by the element detecting unit 142 in a city or the like (S3). Specifically, the position specifying unit 143 specifies the position indicated by the position information included in the line-of-sight data corresponding to the detected element as the position of the element.

  Note that the position specifying unit 143 may acquire, from the wearable terminal 2, information in which position information (shooting position) of the image is associated with each image included in the moving image. In this case, the position specifying unit 143 may specify the position of the element based on the shooting position of the image associated with the image including the element detected by the element detecting unit 142 among the images included in the moving image. Good.

  The position specifying unit 143 may adjust the position of the element based on the terminal position included in the line-of-sight data and the user's viewing direction. For example, since the image shows a landscape in the viewing direction of the user, the elements included in the image are not in the position indicated by the line-of-sight data but in the viewing direction of the user. For this reason, the position specifying unit 143 may adjust the position of the element from the position indicated by the specified position information to a position corresponding to the line-of-sight direction based on the viewing direction included in the line-of-sight data.

  In addition, when the position information of each of the images included in the moving image cannot be specified and only the position where the shooting of the moving image can be specified, the position specifying unit 143 is based on a plurality of continuous images included in the moving image. The position of the element may be specified. For example, the position specifying unit 143 analyzes a plurality of continuous images, specifies the moving direction of the user, and corresponds to the image in which the element is detected based on the moving direction and the position where the shooting of the moving image is started. Specify the position to perform. And the position specific | specification part 143 specifies the position of an element based on the said position.

  FIG. 7 is a diagram illustrating an example of specifying the position of an element based on a plurality of images. As shown in FIG. 7, it is assumed that four continuous images (A) to (D) are included in the moving image. For example, the position specifying unit 143 analyzes the images (A) to (D) and detects objects and contour lines included in the respective images as shown in (a) to (d). For example, in FIGS. 7A to 7D, as a result of analyzing the images (A) to (D), it can be confirmed that two contour lines and the objects X to Z are detected. Then, the position specifying unit 143 specifies the moving direction and moving speed of the user based on the position of the object and the contour line in each of the plurality of images, and the position where the shooting of the moving image is started, the specified moving direction, Based on the specified moving speed, the position corresponding to each of the plurality of images is specified.

[Mental model creation]
Subsequently, the creation unit 144 creates a mental model indicating a map that the user envisions for a city or the like based on the element detected by the element detection unit 142 and the position of the element specified by the position specifying unit 143 ( S4). For example, the creation unit 144 draws an image indicating the element detected by the element detection unit 142 on the plain drawing based on the position of the element specified by the position specification unit 143 as shown in FIG. A simple mental model.

  As shown in FIG. 9, the creation unit 144 includes a spatial route such as a city, and the position of the element specified by the position specification unit 143 with respect to an electronic map (white map) to which no information such as an element is attached. Based on the above, a mental model may be created by drawing an image indicating the element detected by the element detection unit 142.

In addition, the creation unit 144 does not create the map itself, but associates each of the plurality of elements detected by the element detection unit 142 with the position information indicating the position of the element specified by the position specification unit 143. Alternatively, it may be generated as a mental model.
The creation unit 144 creates a 2D map model as a mental model, but may create a 3D map model.

FIG. 10 is a diagram illustrating an example of creating a three-dimensional mental model. First, the element detection unit 142 estimates a geometric coordinate axis from the composition or figure in the image based on the image (a). At the start of the search, the spatial localization is insufficient, and the element and the direction in which the element exists are ambiguous.
Subsequently, it is assumed that the element detection unit 142 detects that the user has continuously confirmed a region indicating a road as an object in the image (b). In this case, a geometric coordinate axis in which the user feels an internal phase with respect to the object is recognized, and a path is generated in the mental model.

Subsequently, it is assumed that the element detection unit 142 detects that the user gazes at an area indicating a road intersection as an object in the image (c). In this case, a geometric coordinate axis in which the user feels an external phase with respect to the object is recognized, and a node is generated in the mental model.
Subsequently, in the image (d), it is assumed that the user looks around a building that exists beside the road as an object. In this case, a geometric coordinate axis in which the user feels an external phase with respect to the object is recognized, and a district is generated in the mental model.

Subsequently, in the image (e), it is assumed that the user looks around an area indicating a road as an object. In this case, a geometric coordinate axis in which the user feels an internal phase with respect to the object is recognized, and a path is generated in the mental model.
Subsequently, in the image (f), it is assumed that the user gazes at a building existing ahead of the road identified as a path in the image (e) as an object. In this case, a geometric coordinate axis in which the user feels an external phase with respect to the object is recognized, and a landmark is generated in the mental model.

  Subsequently, in the image (g), it is assumed that the user confirms a building that can be viewed as a target from between the buildings. In this case, a landmark is generated in the mental model at a position corresponding to the confirmed position. By applying a three-dimensional model, it is possible to generate data that better matches the search behavior.

  For example, when a user detects a building or the like that exists in the distance as a landmark or the like, the two-dimensional mental model may not be able to represent the landmark, but by using a three-dimensional mental model, Buildings that exist far away can also be reflected in the mental model. Thereby, the designer can evaluate the influence degree in the city etc. of the said landmark etc. by confirming the recognition condition of the said landmark etc. of a some user with a three-dimensional mental model, for example.

[Calculation of awareness]
Subsequently, based on the mental model created by the creation unit 144 and a map (reference information) showing the actual elements corresponding to the spatial route such as cities and the positions of the elements (reference information), the calculation unit 145 The user's degree of recognition with respect to the city, etc., showing the ease of understanding, is calculated (S5).

  Specifically, the calculation unit 145 calculates the degree of recognition based on the degree of coincidence between the elements and element positions included in the mental model and the elements and element positions included in the actual map as illustrated in FIG. To do. More specifically, the calculation unit 145 compares the elements and element positions included in the mental model with the elements and element positions included in the actual map. Then, the calculation unit 145 calculates the detection rate of the element by the element detection unit 142 with respect to the element included in the actual map, and the error between the position of the element included in the actual map and the specified position. Then, the calculation unit 145 calculates the degree of recognition based on at least one of the calculated detection rate and error.

Here, the calculation unit 145 may calculate the degree of coincidence with the elements included in the actual map by limiting to only the elements of the path and the node among the elements included in the mental model. For example, when a three-dimensional mental model is created, if the comparison target map is two-dimensional, there is a problem that the degree of coincidence with elements included in the actual map cannot be calculated with high accuracy. Therefore, the calculation unit 145 corrects the notation in the mental model to correct the mental model limited to only the elements of the path and the node, and calculates the degree of coincidence between the corrected mental model and the actual map. May be. FIG. 12 is a diagram illustrating an example of simplifying the mental model. 12A to 12C are front views of a three-dimensional mental model. First, the calculation unit 145 identifies a path and an edge corresponding to the path from the mental model illustrated in FIG. 12A, and draws a line indicating the path as illustrated in FIG. Thereafter, as illustrated in FIG. 12B, the calculation unit 145 simplifies the mental model by deleting elements other than the drawn lines from the three-dimensional mental model.
The calculation unit 145 causes the display unit 12 to display information indicating the calculated degree of recognition, for example. Note that the calculation unit 145 may store information indicating the calculated degree of recognition in the storage unit 13.

[Calculation of complexity]
Subsequently, the calculation unit 145 calculates the complexity indicating the complexity of the city or the building based on the number of elements detected by the element detection unit 142. For example, the number of elements detected per unit time or unit distance is specified in advance as a reference value for each moving image showing a plurality of cities or the like. Then, the calculation unit 145 calculates the number of elements detected per unit time in the video based on the length of the video acquired by the acquisition unit 141 and the number of elements detected by the element detection unit 142. The complexity may be calculated based on the number of elements and the reference value.

  Further, a mental model (collective intelligent map) in which elements detected by the element detection unit 142 corresponding to each of a plurality of users are reflected on an actual map may be generated. Then, the calculation unit 145 may calculate the complexity based on the ratio of the elements detected by the element detection unit 142 to the elements included in the mental model.

[Effect in the first embodiment]
As mentioned above, according to 1st Embodiment, since the recognition degree calculation apparatus 1 detects the element estimated that the user recognized in the city etc. based on the acquired moving image and the pattern of the movement of a gaze, It can be detected that the user perceives an image of a city or the like.

  Then, the recognition level calculation device 1 creates a mental model indicating a map that the user envisions for a city or the like based on the element detected by the element detection unit 142 and the position of the element specified by the position specifying unit 143. Based on the mental model and the reference information indicating the actual elements and the positions of the elements, the user's degree of recognition of the city or the like is calculated. By doing in this way, the degree-of-recognition calculation apparatus 1 can index the size of the image that the user has with respect to the city or the like, that is, the user's intelligibility with respect to the city or the like.

  Thereby, the designer of a city etc. can evaluate whether a city etc. are easy to understand for a user based on the recognition degree which the recognition degree calculation apparatus 1 calculated. Then, the designer can apply or apply the structure of a city or the like evaluated as easy to understand to the user to the design of a new city or the like.

  For example, a map creator can, for example, give a user a map and search for a city, and can create a map that is easy for the user to understand based on the recognition degree of the city obtained as a result. Thereby, the creator of the map can, for example, develop a map notation that is generally easy to understand. The architect of the building, for example, gives the user a floor plan of the building and searches the inside of the building, and the floor plan is easy for the user to understand based on the recognition degree of the resulting building. It can be evaluated whether or not. Thereby, the designer of a building can develop the notation of the floor plan etc. which are generally easy to understand, for example.

  Further, by notifying the user of the degree of recognition, it is possible to recognize how much the user understands the city or the like. And, for example, for users with low awareness, the map created by the map creator is handed over to search the city, or the floor plan created by the building designer is handed over the interior of the building. By searching, in a city or the like, the user can acquire and search for elements in a balanced manner, and the user's physiological direction sense and direction sense can be improved.

  Moreover, the recognition level calculation apparatus 1 calculates the complexity which shows the complexity of a city or a building based on the number of detected elements. Traditionally, the complexity of cities and buildings related to the uniqueness of the user's experience with cities and buildings has not been indexed, so designers can see what design is highly unique to the experience. There was a problem that it was not possible to fully design with awareness of the situation. For example, the complexity that users feel about cities, etc. and the uniqueness of the experience are not indexed, so it is not possible to evaluate design plans from the perspective of complexity when designing designs for cities, public buildings, etc. There was a problem that the proposed plan was difficult to agree. On the other hand, since the degree-of-recognition calculation apparatus 1 can calculate the complexity, the designer evaluates how complex the city indicated by the video is with respect to other cities, etc. It can be applied to the design of high-quality cities and the like. In addition, the designer can design a city or the like in which the user can feel the complexity and uniqueness.

  In the first embodiment, the calculation unit 145 calculates the degree of recognition using the mental model and the actual map illustrated in FIG. 11 as reference information. However, as a result of calculating the degree of recognition of a plurality of users, There may be a problem that the user's recognition level is too low or too high. Therefore, the control unit of the recognition level calculation device 1 is configured such that the element detection unit 142 includes a plurality of users based on the recognition levels of a plurality of users or the detection levels of the respective elements of the plurality of users as illustrated in FIG. A mental model (collective intelligence map) may be created in which elements detected corresponding to each of these are reflected in an actual map. Then, the calculation unit 145 recognizes based on the elements and element positions included in the mental model and the elements and element positions included in the mental model in which elements detected by a plurality of users are reflected on an actual map. May be calculated.

<Second Embodiment>
[Obtain gaze data based on computer graphic animation]
Next, the second embodiment will be described. In the second embodiment, the wearable terminal 2 does not detect a line of sight when a user passes through an actual space route in a city or the like, and displays a computer graphic moving image (hereinafter referred to as a CG moving image) indicating the city or the like. It differs from the first embodiment in that the line of sight when the user visually recognizes is detected. Hereinafter, a different part from 1st Embodiment is demonstrated. The description of the same parts as those in the first embodiment will be omitted as appropriate.

  In the second embodiment, a user wearing the wearable terminal 2 visually recognizes a CG video showing a city or the like displayed on a display unit of a video display device (not shown).

  The line-of-sight detection unit 223 of the wearable terminal 2 detects the user's line of sight with respect to the CG moving image. The line-of-sight detection unit 223 generates line-of-sight data in which the time when the line of sight is detected and the line-of-sight position indicating the coordinates of the line of sight in the CG moving image are generated and stored in the storage unit 21. Here, the time when the line of sight is detected is associated with the reproduction time of the CG video.

  The acquisition unit 141 of the recognition degree calculation device 1 acquires a CG moving image indicating a spatial route such as a city and a movement pattern of a user's line of sight corresponding to the CG moving image. Specifically, the acquisition unit 141 acquires a CG video from the video display device, and acquires line-of-sight data corresponding to the CG video from the wearable terminal 2. And the acquisition part 141 acquires a user's gaze movement pattern based on gaze data.

The element detection unit 142 detects an element estimated to be recognized by the user in a city or the like based on the CG moving image acquired by the acquisition unit 141 and the line-of-sight movement pattern.
The position specifying unit 143 specifies the position of the element detected by the element detecting unit 142 in a city or the like. Specifically, the position specifying unit 143 specifies line-of-sight detection time associated with the line-of-sight position when the element is detected with reference to the line-of-sight data. And the position specific | specification part 143 specifies the reproduction | regeneration time of the CG moving image corresponding to the detection time of the specified eyes | visual_axis. Since position information indicating each position in a city or the like is associated with a plurality of reproduction times of the CG video, the position specifying unit 143 determines the position indicated by the position information associated with the reproduction time. Specify the position of the element.

  Subsequently, the creation unit 144 creates a mental model of the user based on the elements detected by the element detection unit 142 and the positions of the elements specified by the position specification unit 143. Based on the mental model created by the creation unit 144 and the map (reference information) indicating the elements included in the cities and the like corresponding to the CG video and the positions of the elements (reference information), the calculation unit 145 determines the degree of recognition of the user's cities and the like. calculate.

[Effects of Second Embodiment]
As described above, according to the second embodiment, the degree-of-recognition calculation apparatus 1 is based on a CG moving image indicating a city or the like and a line-of-sight movement pattern corresponding to the CG moving image. An element that is assumed to be recognized by the user is detected. By indicating the CG video as a planning stage or a city under development, the recognition level calculation device 1 detects an element estimated to be recognized by the user for the city and the like, and calculates the level of recognition. be able to. Therefore, a designer in a planning stage or a city under development can evaluate whether it is easy for the user to understand during the planning stage or development in the city or the like.

  In the second embodiment, the wearable terminal 2 acquires the line-of-sight data when the user visually recognizes the CG video, but the present invention is not limited to this. For example, the wearable terminal 2 may acquire line-of-sight data when a user visually recognizes a moving image obtained by photographing a city or the like in advance. By doing in this way, the recognition degree calculation apparatus 1 can calculate the recognition degree of the user with respect to the city or the like even if the user does not go to a place where the city or the like is.

  Further, the CG moving image is a moving image indicating a predetermined spatial route in a city or the like, but is not limited thereto. For example, the CG moving image may be a moving image indicating a place where the user cannot actually pass, such as in the air. By doing in this way, the user who has visually recognized the CG video can recognize an image of a city or the like that is not normally recognized.

  The CG movie is, for example, a VR (Virtual Reality) format or an AR (Augmented Reality) format CG movie that can display an arbitrary space of a pre-designed three-dimensional space model. May be. For example, in response to receiving instruction information from the user via a controller or the like for instructing the direction that the user wants to confirm, in response to detecting a moving image of a format that changes CG or the user's line of sight, A moving image in which the CG is changed corresponding to the line of sight, or a format in which the CG does not change and is stored as a log in association with the instruction information from the user and the shape of the three-dimensional CG, and can be confirmed later May be a video. By doing in this way, since the recognition degree calculation device 1 allows the user to visually recognize one element from various directions, it becomes easy to specify the three-dimensional model corresponding to the element visually recognized by the user. Detection accuracy can be improved.

<Third Embodiment>
[Acquires a line-of-sight movement pattern based on a video taken by the wearable terminal 2]
Subsequently, a third embodiment will be described. In the third embodiment, the wearable terminal 2 generates only the moving image without generating the line-of-sight data, and the recognition degree calculation device 1 displays the movement pattern of the user's line of sight based on the moving image acquired from the wearable terminal 2. It differs from the first embodiment in that it is acquired. In the third embodiment, it is assumed that a terminal that does not have a function of generating line-of-sight data is used as the wearable terminal 2.

In the third embodiment, the acquisition unit 141 acquires, from the wearable terminal 2, a moving image that indicates a spatial route of a city or the like photographed by the wearable terminal 2.
The element detection unit 142 detects an element estimated to be recognized by the user in a city or the like based on the composition corresponding to each of the plurality of images included in the moving image. Specifically, the element detection unit 142 identifies a composition corresponding to the image by analyzing each of the plurality of images included in the moving image. Then, the element detection unit 142 refers to the element specifying information illustrated in FIG. 6A or 6B and specifies an element associated with the specified composition, thereby detecting an element that is estimated to be recognized by the user.

The position specifying unit 143 specifies the position of the detected element in the city or the like. Specifically, in the third embodiment, the terminal position acquired by the position information acquisition unit 221 is associated with the playback position of the moving image. The position specifying unit 143 specifies the playback position of the moving image based on the image when the element is detected, and specifies the terminal position associated with the playback position as the position of the detected element.
The functions of the creation unit 144 and the calculation unit 145 are the same as those in the first embodiment, and a description thereof will be omitted.

[Effect in the third embodiment]
As described above, according to the third embodiment, the recognition degree calculation device 1 can identify an element that is estimated to be recognized by a user in a city or the like based only on a moving image, and thus obtains user's line-of-sight data. Even if it is not possible, it is possible to calculate the user's degree of recognition of the city.

  As mentioned above, although this invention was demonstrated using embodiment, the technical scope of this invention is not limited to the range as described in the said embodiment. It will be apparent to those skilled in the art that various modifications or improvements can be added to the above embodiment.

  For example, in the above-described embodiment, the degree-of-recognition calculation apparatus 1 acquires the moving image and line-of-sight data from the wearable terminal 2 through communication, but the present invention is not limited to this. For example, an external storage medium removed from the wearable terminal 2 may be attached to the recognition degree calculation device 1, and the recognition degree calculation device 1 may acquire moving images and line-of-sight data from the external storage medium.

  In the above-described embodiment, the recognition level calculation device 1 and the wearable terminal 2 are different devices, but the recognition level calculation device 1 and the wearable terminal 2 may be integrated devices. . For example, the wearable terminal 2 may have a function included in the recognition degree calculation device 1, or the recognition degree calculation device 1 may have a function included in the wearable terminal 2. Here, when the wearable terminal 2 has a function included in the recognition degree calculation device 1, the input unit of the wearable terminal 2 may be configured by a button, a sensor terminal, or the like. Then, the line of sight input to the wearable terminal 2 or the gesture input by the user's hand or the like in the range where the wearable terminal 2 can be photographed (within the user's field of view) may be received by the button or the sensor terminal.

DESCRIPTION OF SYMBOLS 1 ... Recognition degree calculation apparatus, 11 ... Input part, 12 ... Display part, 13 ... Memory | storage part, 14 ... Control part, 141 ... Acquisition part, 142 ... Element detection Part, 143 ... position specifying part, 144 ... creation part, 145 ... calculation part, 2 ... wearable terminal, 21 ... storage part, 22 ... control part, 221 ... position Information acquisition unit, 222 ... photographing unit, 223 ... gaze detection unit, 224 ... transmission unit

Claims (13)

An acquisition unit that acquires a moving image showing a spatial route of a city or a building and a movement pattern of a user's line of sight corresponding to the moving image;
An element detection unit that detects an element that is estimated to be recognized by the user in the city or the building based on the moving image and the movement pattern of the line of sight;
A position specifying unit for specifying a position of the detected element in the city or the building;
A creation unit that creates a mental model indicating a map that the user envisions for the city or the building based on the detected element and the position of the identified element;
The city of the user indicating the ease of understanding of the city or the building to the user based on the mental model and the reference information indicating the actual element corresponding to the spatial path and the position of the element Or a calculation unit for calculating the degree of recognition for the building;
A recognition degree calculation device comprising: The calculation unit calculates the degree of recognition based on the degree of coincidence between the element and the position of the element included in the mental model and the element and the position of the element included in the reference information.
The recognition degree calculation apparatus according to claim 1. The calculation unit calculates a complexity indicating the complexity of the city or the building based on the number of the elements detected by the element detection unit.
The recognition degree calculation apparatus according to claim 1 or 2. The element detection unit detects the element based on a composition corresponding to an image included in the moving image and a movement pattern of the line of sight;
The degree-of-recognition calculation apparatus according to any one of claims 1 to 3. The position specifying unit specifies a position of the element based on a shooting position of the image associated with an image included in the moving image;
The degree-of-recognition calculation apparatus according to any one of claims 1 to 4. The position specifying unit specifies a position corresponding to an image in which the element is detected as a position of the element based on a plurality of continuous images included in the moving image.
The degree-of-recognition calculation apparatus according to any one of claims 1 to 4. The acquisition unit acquires a computer graphic video indicating the city or a building as the video.
The degree-of-recognition calculation apparatus according to any one of claims 1 to 6. The acquisition unit acquires, as the moving image, a moving image shot by a wearable terminal attached to the user.
The degree-of-recognition calculation apparatus according to any one of claims 1 to 5. The element detection unit identifies a composition pattern of an image included in a moving image captured by the wearable terminal, and further detects the element based on the composition pattern of the image;
The recognition degree calculation device according to claim 8. The acquisition unit acquires a movement pattern of the user's line of sight detected by a line-of-sight detection device that detects the line of sight of the user.
The recognition degree calculation apparatus according to any one of claims 1 to 7. An acquisition unit that acquires a moving image that is captured by a wearable terminal attached to a user and that indicates a spatial route of a city or a building;
An element detection unit that detects an element that is estimated to be recognized by the user in the city or the building based on a composition corresponding to each of a plurality of images included in the video;
A position specifying unit for specifying a position of the detected element in the city or the building;
Based on the detected element and the identified position of the element, a creation unit that creates a mental model indicating a map that the user envisions for the city or the building;
Based on the mental model and the reference information indicating the actual position of the element and the position of the element in the area corresponding to the spatial path, the user or the user can easily understand the city or the building. A calculation unit for calculating the degree of recognition for the city or the building;
A recognition degree calculation device comprising: Executed by the computer,
Acquiring a moving image showing a spatial path of a city or a building, and a movement pattern of a user's line of sight with respect to the moving image;
Detecting an element presumed to be recognized by the user in the city or the building based on the moving image and the movement pattern of the line of sight;
Detecting the position of the detected element in the city or the building;
Creating a mental model showing a map envisioned by the user for the city or the building based on the detected element and the position of the element;
The city of the user indicating the ease of understanding of the city or the building to the user based on the mental model and the reference information indicating the actual element corresponding to the spatial path and the position of the element Or calculating the degree of recognition for the building;
A recognition calculation method comprising: Computer
An acquisition unit that acquires a moving image showing a spatial route of a city or a building, and a movement pattern of a user's line of sight corresponding to the moving image,
An element detection unit that detects an element that is estimated to be recognized by the user in the city or the building based on the moving image and the movement pattern of the line of sight,
A position specifying unit for specifying the position of the detected element in the city or the building;
Based on the detected element and the position of the identified element, a creation unit that creates a mental model indicating a map that the user envisions for the city or the building, and the mental model, The user's perception of the city or the building, which indicates the ease of understanding of the city or the building to the user based on the actual element corresponding to the spatial path and the reference information indicating the position of the element A calculation unit for calculating the degree,
Recognition calculation program to function as.