KR102542977B1 - Device and method for evaluation of sign visibility based on mr simulation - Google Patents

Abstract

An electronic device according to an embodiment of the present disclosure can generate an MR-based visibility test simulation by matching at least two test signs to specific positions in a video captured while moving along a walking path, acquire the gaze data of a user through an eye-tracking sensor while playing each MR-based visibility test simulation through a head-mounted display; and compare visibility parameters for at least two test signs based on each of the gaze data.

Description

Apparatus and method for evaluating sign visibility based on MR simulation {DEVICE AND METHOD FOR EVALUATION OF SIGN VISIBILITY BASED ON MR SIMULATION}

It relates to an apparatus and method for evaluating sign visibility based on MR simulation, and more specifically, to an electronic device that evaluates sign visibility by acquiring eye tracking data of a sign through MR simulation.

Mixed Reality (MR) is a technology that combines the information of the real world and the virtual world to create a space that fuses the two worlds, integrating the advantages of augmented reality (AR) and virtual reality (VR). Mixed reality is an improvement of augmented reality, and augmented reality is a technology that overlays a real world that a user sees with eyes and a virtual world having additional information to show as a single image.

A head mounted display (HMD) for a virtual reality display is a means for providing a virtual reality image by being worn on a user's head and displaying a 3D image directly to the user's eyes.

According to embodiments, an electronic device for generating MR-based test simulation for visibility evaluation of various types of sign designs may be provided.

According to embodiments, an electronic device may be provided that collects gaze data of a user while reproducing an MR-based test simulation, calculates a gaze entropy value, and compares visibility of various types of sign designs.

An electronic device according to an aspect of an embodiment includes a display, an eye tracking sensor, a memory for storing instructions, and at least one processor, wherein the at least one processor takes pictures while moving along a first path. At least two visibility test simulations matching at least two test signs to a first position of a video are generated, and while the at least two visibility test simulations are reproduced through the display, the user's respective user information is displayed through the eye tracking sensor. Obtain gaze data of , and compare visibility parameters for the at least two test signs based on each gaze data.

A method for evaluating sign visibility based on MR simulation according to another aspect of an embodiment includes selecting a first base scene photographed by a camera for a predetermined time along a first path according to a test situation from among a plurality of base scenes. generating at least two visibility test simulations by matching at least two test signs with the first base scene, respectively, acquiring gaze data of the user while reproducing the at least two visibility test simulations, respectively; The method may include comparing visibility parameters of the at least two test signs based on the gaze data, and selecting a sign having the highest visibility among the at least two test signs.

A recording medium storing computer readable instructions according to another aspect of the embodiment, wherein the instructions, when executed by an electronic device, cause the electronic device to perform: a first command according to a test situation among a plurality of base scenes; Selecting a first base scene photographed by a camera for a predetermined time along a route, matching at least two test signs to the first base scene, and generating at least two visibility test simulations. Obtaining each user's gaze data while reproducing each visibility test simulation, comparing visibility parameters of the at least two test signs based on the respective gaze data, and performing the operation of comparing the visibility parameters of the at least two test signs. An operation of selecting a sign with the highest visibility among them may be included.

Other aspects, features, and advantages other than those described above will become apparent from the following drawings, claims, and detailed description of the invention.

According to the embodiments, in order to evaluate the visibility of various types of existing sign designs, actual signs are produced, or existing signs are removed and test signs are installed so that pedestrians do not subjectively judge visibility. While viewing the MR-based visibility test simulation through the head-mounted display, data on the user's gaze may be acquired to objectively determine the degree of visibility.

According to embodiments, a test sign having the highest visibility may be provided by comparing visibility parameter values that may be calculated as objective indicators through a simple simulation with respect to a plurality of test signs.

1 shows a block diagram of an electronic device according to an embodiment.
2 shows a conceptual diagram of a visibility test simulation according to an embodiment.
3 shows a flow chart of a method of generating a visibility test simulation according to one embodiment.
4 illustrates components of a visibility test simulation according to an embodiment.
5 illustrates an example of an AOI in a visibility test simulation image according to an embodiment.
6 illustrates an evaluation analysis graph of gaze tracking data for a visibility test simulation according to an exemplary embodiment.
7 is a flowchart of an optimal sign selection method using visibility test simulation according to an embodiment.

Hereinafter, embodiments will be described in detail with reference to the accompanying drawings. However, the scope of rights is not limited or limited by these embodiments. Like reference numerals in each figure indicate like elements.

The terms used in the description below have been selected as general and universal in the related technical field, but there may be other terms depending on the development and / or change of technology, the preference of customary technicians, etc. Therefore, terms used in the following description should not be understood as limiting technical ideas, but should be understood as exemplary terms for describing embodiments.

In addition, in certain cases, there are also terms arbitrarily selected by the applicant, and in this case, the detailed meaning will be described in the corresponding description section. Therefore, terms used in the following description should be understood based on the meaning of the term and the contents throughout the specification, not simply the name of the term.

1 shows a block diagram of an electronic device according to an embodiment.

The electronic device 100 according to an embodiment may include a processor 110, a memory 120, a display 130, and a sensor 140. The processor 110 may control the memory 120 , the display 130 and the sensor 140 . For example, processor 110 may be referred to as an application processor and/or processor. Hereinafter, the operation of the electronic device 100 may be understood as being performed by the processor 110 of the electronic device 100 . In various embodiments, some of the components included in the electronic device 100 may be configured as a separate device. For example, the sensor 140 may be configured as a separate device (eg, the HMD device 205).

The processor 110 may be one or more of a central processing unit (CPU), an application processor (AP), a graphics processing unit (GPU), an image signal processor (ISP) of a camera, or a communication processor (CP). may include more. According to an embodiment, the processor 120 may be implemented as a system on chip (SoC) or system in package (SiP). The processor 120 may control at least one other component (eg, hardware or software component) of the electronic device 100 connected to the processor 120 by driving an operating system or an application program, and , various data processing and calculations can be performed. The processor 120 may load and process a command or data received from at least one of the other components (eg, a sensor) into a volatile memory, and store resultant data in a non-volatile memory.

In one embodiment, the electronic device 100 may output an MR image through the display 130. The MR image may be generated based on a 360-degree image actually captured and may include one or more virtual objects. For example, the virtual object may be a pedestrian guidance sign. The MR image may be an image of a predetermined length moving along a certain path to represent a walking situation. In various embodiments, a visibility test simulation may be generated by matching a sign object to an MR image.

The memory 120 may include volatile memory or non-volatile memory. The volatile memory 832 may be composed of, for example, random access memory (RAM) (eg, DRAM, SRAM, or SDRAM). Non-volatile memory includes, for example, programmable read-only memory (PROM), one time PROM (OTPROM), erasable PROM (EPROM), electrically EPROM (EEPROM), mask ROM, flash ROM, flash memory, hard disk (HDD) drive), or a solid state drive (SSD). In addition, the non-volatile memory, depending on the connection type with the electronic device 100, the built-in memory 120 disposed therein, or a stand-alone type of external memory that can be connected and used only when necessary. Consists of It can be. External memory is a flash drive, for example, compact flash (CF), secure digital (SD), Micro-SD, Mini-SD, extreme digital (xD), multi-media card (MMC), or memory. Can contain sticks. The external memory may be functionally or physically connected to the electronic device 100 through wire (eg, cable or universal serial bus (USB)) or wirelessly (eg, Bluetooth).

In one embodiment, the memory 120 may store a plurality of base scenes. Base scenes can be created in a variety of ways. For example, the base scene may be a real image captured by a camera, an image produced by computer graphics, a 2D image, a 3D image, or an image providing a 360 degree view. A base scene can be determined by various factors, such as topographical features, road characteristics, surrounding congestion, color composition of the surrounding environment, and degree of light sources (sun, street lights, lights from nearby buildings, etc.) that change over time. The base scene may be produced according to a walking situation according to the movement of a pedestrian, and may be produced according to a driving situation according to the movement of a car. The walking situation may be determined by a plurality of factors, such as pedestrian characteristics, walking speed, pedestrian viewing range, pedestrian road, and surrounding environment according to walking time. The driving situation may be determined by a plurality of factors, such as road traffic conditions, driving speed, driver's or passenger's field of view, lane characteristics, and surrounding environment according to driving time. In various embodiments, various base scenes may be created/stored according to an object to be evaluated for visibility and a production situation.

The display 130 may include, for example, a liquid crystal display (LCD), a light emitting diode (LED) display, an organic light emitting diode (OLED) display, a microelectromechanical system (MEMS) display, or an electronic paper display. can The display, according to one embodiment, may be implemented to be flexible, transparent, or wearable. The display includes touch circuitry that can detect a user's touch, gesture, proximity, or hovering input, or a pressure sensor that can measure the amount of pressure applied to a touch (interchangeably a "force sensor"). can do. The touch circuit or pressure sensor may be integrally implemented with the display or implemented as one or more sensors separate from the display.

In one embodiment, the display 130 may output a mixed reality-based image in which virtual content is combined with an image captured using a camera. The display 130 may be included in a head-mounted display (HMD), worn on a head by a user, and output a mixed reality-based 3D 360-degree video directly to a display area close to eyes.

The sensor 140 measures or detects, for example, an internal operating state (eg, power or temperature) or an external environmental state (eg, altitude, humidity, or stepping on) of the electronic device 100, Electrical signals or data values corresponding to measured or sensed state information may be generated. The sensor 140 may be, for example, a gesture sensor, a gyro sensor, an air pressure sensor, a magnetic sensor, an acceleration sensor, a grip sensor, a proximity sensor, a color sensor (eg, an RGB sensor), an IR (infrared) sensor, and a biometric sensor. (e.g. iris sensor, fingerprint sensor, or heartbeat rate monitoring (HRM) sensor, electronic nose sensor, electromyography (EMG) sensor, electroencephalogram (EGG) sensor, electrocardiogram (ECG) sensor), temperature sensor, humidity sensor, An illuminance sensor or UV (ultra violet) sensor may be included. The sensor 140 may further include a control circuit for controlling one or more sensors included therein.

In an embodiment, when the sensor 140 includes an eye tracking sensor, the electronic device 100 recognizes a direction in which the user's eyes are directed through the eye tracking sensor, and generates an MR corresponding to the direction. The coordinates (point) of the image can be determined. While the MR image is being reproduced, the eye tracking sensor may detect movement of the user's pupils, change in pupil size, and blinking of the eyes.

2 shows a conceptual diagram of a visibility test simulation according to an embodiment.

The electronic device 100 according to an embodiment may generate a simulation image for a visibility test and calculate a degree of visibility based on data obtained by tracking a gaze of a user viewing the simulation image. Visibility refers to the ability to recognize shapes and colors. That is, visibility refers to a property in which the shape or color of an object can be easily identified even from a long distance. For example, the greater the difference in brightness, the higher the visibility. Factors affecting visibility include resolution, sharpness, and brightness, and may be expressed as relative values according to environmental factors around the object, not absolute values. For example, an image with high visibility is a high-resolution (or high-definition) image, an image with increased sharpness, a vector graphics image, an image from a short distance, an image captured when the amount of light is temporally abundant (ie, in a state of high ambient light) images), etc.

The electronic device 100 may select one 201a from among a plurality of base scenes 201 obtained by reflecting various terrain features, road environment features, color composition, and the like. The base scene 201 may be defined with various components. For example, topographical specifics (e.g., parks, roads, suburbs, downtowns, etc.), road features (e.g., intersections, straights, etc.), neighborhood congestion (e.g., many pedestrians, traffic jams, etc.), or different color schemes. (e.g. single color, various colors, etc.) The base scene 201 may be, for example, a 360-degree video taken while moving along a certain path representing a walking situation. A certain route may include one or more guide signs 202 for pedestrians. The guide sign 202 for pedestrians may be made of various types of signs composed of various shapes, sizes, and colors. The electronic device 100 may select one 202b from among a plurality of signs 202 . The electronic device 100 may generate the MR image 203 by matching the selected base scene 201a and the sign 202b. The MR image 203 may be used as a visibility test simulation for evaluating the degree of visibility of the sign 202b included in the image.

When the user 210 wears the head mounted display 220, the electronic device 100 reproduces the visibility test simulation 203 generated by matching the sign 202b to the base scene 201a, and the visibility test simulation ( While 203) is reproduced, data 204 tracking the user's gaze may be obtained through the eye tracking sensor 140 . The data 204 tracking the user's gaze includes the data of detecting the movement of the user's pupils, the change in size of the pupils, and blinking of the user's pupils with respect to the sign 202b displayed on the image.

The electronic device 100 may analyze the gaze tracking data 204 to calculate the degree of visibility. In an embodiment, the electronic device 100 analyzes eye tracking data obtained by evaluating visibility through a plurality of simulations generated by matching a plurality of signs 202 to the same base scene 201a, Among the dog signs 202, a sign with the highest visibility may be selected.

3 shows a flow chart of a method of generating a visibility test simulation according to one embodiment. The electronic device 100 according to an embodiment may generate a visibility test simulation for evaluating visibility of a sign.

In step S301, the electronic device 100 may obtain a 360-degree video captured using a camera. The electronic device 100 may select the 360-degree video from a plurality of base scenes stored in the memory 120. In an embodiment, in order to evaluate the visibility of a guide sign provided to pedestrians, the electronic device 100 360-degree video captured by the camera for a certain period of time can be acquired while moving along a certain path according to the walking situation (walking speed, pedestrian view range, pedestrian road, surrounding environment according to the walking time, etc.) including the point where the guide sign will be installed. In another embodiment, in order to evaluate the visibility of the guidance sign provided to the driver, the electronic device 100 includes a point where the guidance sign is to be installed and driving conditions (driving speed, driver's or passenger's field of vision) A 360-degree video captured by a camera may be acquired while moving along a certain path according to a range, lane conditions, surrounding environment according to driving time, etc. In various embodiments, the electronic device 100 may perform a purpose and target of a visibility test. Accordingly, a 360-degree video clip may be acquired by considering the topographical characteristics of the moving route, the road characteristics, the degree of congestion, the color composition of the surrounding environment, and the characteristics of the surrounding environment according to the shooting time.

In step S302, the electronic device 100 may remove the existing sign included in the 360-degree video. In an embodiment, the electronic device 100 designates an area of a specific object in an image in a segment unit using a machine learning-based inpainting technique, and naturally fills the area from which the object is removed to match the surrounding background. You can remove the existing sign object by giving

In step S303, the electronic device 100 may match a test sign for a visibility test to a 360-degree video clip. In an embodiment, the electronic device 100 may match the test signs by determining the installation location, size, color according to the light source, and shadow of the test signs.

4 illustrates components of a visibility test simulation according to one embodiment.

In one embodiment, the electronic device 100 may determine the installation location by considering the location, light source, shadow, etc., so as to match the test sign as a virtual object to the actual captured image. Referring to (a) of FIG. 4 , an installation location may be determined adjacent to a peripheral lane. Referring to (b) of FIG. 4 , the brightness of the test sign may be determined based on the light source (the sun). Referring to (c) of FIG. 4 , the position, size, and color of the shadow of the test sign according to the light source may be determined. The electronic device 100 may match the virtual objects (test signs) with real components so that the user cannot distinguish between real objects and virtual objects even when matching virtual objects (test signs) with real images.

5 illustrates an example of an AOI in a visibility test simulation image according to an embodiment.

In an embodiment, the electronic device 100 may calculate the degree of visibility using data obtained by tracking the gaze of a user watching a 360-degree video. Referring to (a) of FIG. 5, on the screen of a 360-degree video, an area of interest (AOI) is divided as shown in (b), and the area for the visibility evaluation target (sign) can be displayed as an AOI. there is. The electronic device 100 may calculate the degree of visibility of the sign (the 25th area of the AOI) using the AOI shown in (b) of FIG. 5 .

Gaze data can be classified into fixation and saccade. Fixation means the line of sight when a person concentrates on one point, and Saccade means the line of sight that moves between fixations. In addition, eye blinking (Blink) and pupil diameter (pupil diameter) may be included. Among gaze data, Fixation and Saccade have a high correlation with visibility. For example, the electronic device 100 may draw a visual feature map and a scan path using Fixation. The visual feature map is a heat map indicating which points a person is paying attention to, and the scan path is a plot showing how the fixation moved. The electronic device 100 digitizes the visual feature map and the scan path and compares which object (sign) has the best visibility. The electronic device 100 may digitize the visual feature map and the scan path using gaze entropy. Gaze entropy is a criterion for determining how random a gaze movement is. The higher the gaze entropy, the lower the frequency of the gaze staying at one point, and the lower the gaze entropy, the higher the frequency of the gaze being focused on one point. The electronic device 100 may determine that the visibility of the sign is high (good) as the gaze entropy is low. In an embodiment, the electronic device 100 includes five parameters for gaze data: Time to First Fixation (FFT), Total Duration of Fixations (FD), Scene Stationary Gaze Entropy (SSGE), Target Stationary Gaze Entropy (TSGE), Visibility can be judged based on STGE (Scene Transition Gaze Entropy). FFT is the time taken for the user (the subject of the simulation) to see the sign for the first time, FD is the total time the subject's gaze stays on the sign, SSGE is the stationary entropy of the gaze data in the entire image, TSGE is the overall Stationary entropy excluding the time the gaze stays on the sign in the video, STGE means moving entropy in the entire video.

와 전이 시선 엔트로피(transition gaze entropy, TGE)

로 정의할 수 있다. SGE는 특정 지점에 시선이 머문 빈도를 계산하고, TGE는 특정 지점에 시선이 머문 빈도와 함께 다른 지점으로 시선이 이동하는 빈도까지 계산한다. 전체 시뮬레이션에 대한 SGE(Scene Stationary Gaze Entropy, SSGE)와 전체 시뮬레이션 속 표지판에 대한 SGE(Target Stationary Gaze Entropy, TSGE)를 계산할 수 있다.Gaze entropy can be expressed as a value between 0 and 1. The closer to 0 shows a pattern concentrated on one point (fixation), the closer to 1 shows a pattern with a random distribution (distributed pattern). Gaze entropy is stationary gaze entropy (SGE)

and transition gaze entropy (TGE)

can be defined as SGE calculates the frequency of gaze staying at a specific point, and TGE calculates the frequency of gaze moving to another point along with the frequency of gaze staying at a specific point. You can calculate SGE (Scene Stationary Gaze Entropy, SSGE) for the entire simulation and Target Stationary Gaze Entropy (TSGE) for the signs in the entire simulation.

, Fixation 시퀀스

에 대해서,

의 전이 확률

, 정지 확률

은 하기 수학식 1과 같이 계산할 수 있다.AOI set

, Fixation sequence

about,

transition probability of

, the probability of stopping

can be calculated as in Equation 1 below.

는 i번째 AOI에서 j번째 AOI로 이동한 Fixation의 개수,

는 i번째 AOI에 위치한 Fixation의 개수,

은 해당 영상에서 피험자의 전체 Fixation 개수를 나타낸다. 전자 장치(100)는 도 5의 (a)를 수직하게 24등분한 각 영역을 AOI로 설정하고, 표지판이 위치한 공간을 25번째 AOI로 설정할 수 있다. 전자 장치(100)는 정지한 시선 엔트로피(SGE)를 반복 계산하여 SSGE를 계산하고, 전이 시선 엔트로피(TGE)를 계산하여 STGE를 계산한다. TSGE는 하기 수학식 2와 같이 정지한 시선 엔트로피(SGE)를 계산할 때, 표지판 위에 있는 Fixation을 제외하여 계산할 수 있다.

is the number of Fixations moved from the i-th AOI to the j-th AOI,

is the number of Fixations located in the ith AOI,

represents the total number of fixations of the subject in the corresponding image. The electronic device 100 may set each area obtained by dividing (a) of FIG. 5 vertically into 24 equal parts as the AOI, and may set the space where the sign is located as the 25th AOI. The electronic device 100 calculates SSGE by iteratively calculating still gaze entropy (SGE), and calculates STGE by calculating transition gaze entropy (TGE). TSGE can be calculated by excluding the fixation on the sign when calculating the still gaze entropy (SGE) as shown in Equation 2 below.

는 표지판(25번째 AOI)에 위치한 Fixation의 개수를 나타낸다.

represents the number of Fixations located on the sign (the 25th AOI).

6 illustrates an evaluation analysis graph of gaze tracking data for a visibility test simulation according to an embodiment. In an embodiment, the electronic device 100 may determine the degree of visibility of the test sign based on five parameters (FFT, FD, SSGE, TSGE, and STGE) of the gaze data. Visibility is a relative value, and the electronic device 100 may compare parameter values calculated for a plurality of test signs to select a sign with the highest visibility. For example, in order to evaluate the visibility of the four test signs 601 to 604, four visibility test simulations in which the four test signs are matched to the base scene are tested on the user, and the result of analyzing the gaze data is obtained. It can be shown graphically as shown in FIG. 6 . The test signs 601 to 604 were manufactured with different designs, colors, and shapes, respectively. The electronic device 100 may evaluate that the visibility is high (good) as the FFT, SSGE, TSGE, and STGE parameter values of the gaze data are small. In the case of FD, the larger the value, the more noticeable it is, so the inverse of addition (1-FD) is shown in the graph. The electronic device 100 may determine that the fourth test sign 604 having the smallest size of the graph area has the highest visibility among all test signs.

7 is a flowchart of an optimal sign selection method using visibility test simulation according to an embodiment. In an embodiment, the electronic device 100 may set a visibility test simulation according to a sign to be tested and a sign installation environment.

In step S701, the electronic device 100 may select one of the base scenes stored in the memory 120 according to visibility test simulation settings. Visibility test simulation settings may be determined as set values for components such as a test situation, topographical characteristics, road characteristics, surrounding congestion, color composition of the surrounding environment, and light source. For example, with respect to a walking situation, when there are many pedestrians on a sidewalk, when the surrounding environment is mainly green, the light source may be set for the visibility test simulation to be the sun. The electronic device 100 may select a base scene that satisfies simulation settings.

In step S702, the electronic device 100 may generate a test case including a plurality of test target signs. The electronic device 100 may generate a plurality of visibility test simulations by matching a plurality of test target signs to the selected base scene, respectively.

In step S703, the electronic device 100 may acquire user gaze data for each test case. The electronic device 200 may obtain gaze data of the user using the eye tracking sensor 140 while reproducing each visibility test simulation of the test case. The electronic device 200 may obtain a plurality of gaze data for each visibility test simulation by performing a test on a plurality of users.

In step S704, the electronic device 100 may calculate visibility parameters (FFT, FD, SSGE, TSGE, STGE) for a plurality of gaze data of the test case. The electronic device 100 may calculate the visibility parameter value according to the method described above with reference to FIGS. 5 and 6 .

In step S705, the electronic device 100 may compare visibility parameter values of a plurality of test target signs and select a sign with the highest visibility as a final sign.

Various embodiments of this document and terms used therein are not intended to limit the technology described in this document to a specific embodiment, and should be understood to include various modifications, equivalents, and/or substitutes of the embodiments. In connection with the description of the drawings, like reference numerals may be used for like elements. Singular expressions can be plural unless the context clearly indicates otherwise. In this document, expressions such as “A or B”, “at least one of A and/or B”, “A, B or C”, or “at least one of A, B and/or C” refer to all of the items listed together. Possible combinations may be included. Expressions such as “first,” “second,” “first,” or “second” may modify the elements in any order or importance, and are used only to distinguish one element from another. The components are not limited. When a (e.g., first) element is referred to as being "(functionally or communicatively) coupled to" or "connected to" another (e.g., second) element, that element refers to the other (e.g., second) element. It may be directly connected to the component or connected through another component (eg, a third component).

In this document, "adapted to or configured to" means "adapted to or configured to" depending on the situation, for example, hardware or software "adapted to," "having the ability to," "changed to," ""made to," "capable of," or "designed to" can be used interchangeably. In some contexts, the expression "device configured to" can mean that the device is "capable of" in conjunction with other devices or components. For example, the phrase "a processor set up (or configured) to perform A, B, and C" may include a dedicated processor (eg, an embedded processor) to perform those operations, or one stored in a memory device (eg, memory 120). By executing the above programs, it may mean a general-purpose processor (eg, CPU or AP) capable of performing corresponding operations.

The term "module" used in this document includes a unit composed of hardware, software, or firmware, and may be used interchangeably with terms such as logic, logic blocks, parts, or circuits, for example. can A “module” may be an integrally constructed component or a minimal unit or part thereof that performs one or more functions. A "module" may be implemented mechanically or electronically, for example, a known or future developed application-specific integrated circuit (ASIC) chip, field-programmable gate arrays (FPGAs), or A programmable logic device may be included.

At least some of the devices (eg, modules or functions thereof) or methods (eg, operations) according to various embodiments are instructions stored in a computer-readable storage medium (eg, the memory 120) in the form of program modules. can be implemented as When the command is executed by a processor (eg, the processor 110), the processor may perform a function corresponding to the command. Computer-readable recording media include hard disks, floppy disks, magnetic media (e.g. magnetic tape), optical recording media (e.g. CD-ROM, DVD, magneto-optical media (e.g. floptical disks), built-in memory, etc.) The instruction may include code generated by a compiler or code executable by an interpreter.

Each component (eg, module or program module) according to various embodiments may be composed of a single object or a plurality of entities, and some sub-components among the aforementioned corresponding sub-components may be omitted, or other sub-components may be used. can include more. Alternatively or additionally, some components (eg, modules or program modules) may be integrated into one entity and perform the same or similar functions performed by each corresponding component prior to integration. Operations performed by modules, program modules, or other components according to various embodiments are executed sequentially, in parallel, repetitively, or heuristically, or at least some operations are executed in a different order, omitted, or other operations. this may be added.

100: electronic device
110: processor
120: memory
130: display
140: sensor
210: user
220: HMD

Claims (15)

In electronic devices,
display;
eye tracking sensor;
memory for storing instructions; and
including at least one processor, wherein the at least one processor comprises:
generating at least two visibility test simulations by matching at least two test signs to a first position of a video captured while moving along a first path, respectively;
Obtain gaze data of each user through the eye tracking sensor while reproducing the at least two visibility test simulations through the display; and
comparing visibility parameters for the at least two test signs based on the respective gaze data;
The visibility parameter is the time it takes for the user to see the test sign for the first time (Time to First Fixation), the total time the user's gaze stays on the test sign (Total Duration of Fixations), and the gaze data stop in the entire simulation. At least one of entropy (Scene Stationary Gaze Entropy), stop entropy excluding the time the gaze stayed on the sign in the entire simulation (Target Stationary Gaze Entropy), and transition entropy in the entire simulation (Scene Transition Gaze Entropy) An electronic device.
According to claim 1,
The at least one processor determines the size of the test sign at the first location by considering at least one of a size of the test sign, a light source related to the first position, brightness of the test sign according to the light source, or a size of a shadow of the test sign. Electronic device, matching test signs.
According to claim 1,
The visibility parameter is defined as a gaze entropy value, and the gaze entropy is determined by fixation and saccade of the gaze data.
delete According to claim 1,
Wherein the at least one processor calculates the visibility parameter by calculating the gaze entropy for a plurality of regions of interest (AOIs) in which the screen of the video is divided into regions of a certain area and a region of interest for a test sign.
According to claim 1,
The electronic device, wherein the display and the eye tracking sensor are in the form of a Head Mounted Display (HMD).
According to claim 1,
The video is a 3D video providing a 360-degree view, and the visibility test simulation is generated based on mixed reality (MR).
In the method for evaluating sign visibility based on MR simulation,
selecting a first base scene captured by a camera for a predetermined time along a first path according to a test situation from among a plurality of base scenes;
generating at least two visibility test simulations by matching at least two test signs with the first base scene, respectively;
acquiring gaze data of a user while reproducing the at least two visibility test simulations, respectively;
comparing visibility parameters for the at least two test signs based on the respective gaze data; and
Selecting a sign with the highest visibility among the at least two test signs
including,
The visibility parameter is the time it takes for the user to see the test sign for the first time (Time to First Fixation), the total time the user's gaze stays on the test sign (Total Duration of Fixations), and the gaze data stop in the entire simulation. At least one of entropy (Scene Stationary Gaze Entropy), stop entropy excluding the time the gaze stayed on the sign in the entire simulation (Target Stationary Gaze Entropy), and moving entropy in the entire simulation (Scene Transition Gaze Entropy) Method.
According to claim 8,
Generating the at least two visibility test simulations,
Matching the test sign to the first position in consideration of at least one of a size of the test sign, a light source, brightness of the test sign according to the light source, or a size of a shadow of the test sign.
According to claim 8,
The method of claim 1 , wherein the visibility parameter is defined as a gaze entropy value, and the gaze entropy is determined by fixation and saccade of the gaze data.
delete According to claim 8,
Comparing the visibility parameters,
Calculating the visibility parameter by calculating the gaze entropy for a plurality of regions of interest (AOIs) in which the screens of each of the at least two visibility test simulations are divided into regions of a certain area and the region of interest for the test sign, respectively. Method.
According to claim 8,
The acquiring of each user's gaze data includes reproducing the visibility test simulation through a head mounted display (HMD) worn by the user, and collecting the user's gaze data through an eye tracking sensor.
According to claim 8,
The base scene is a 3D video providing a 360 degree view, and the visibility test simulation is generated based on mixed reality (MR).
A recording medium storing computer readable instructions, which when executed by an electronic device cause the electronic device to:
selecting a first base scene captured by a camera for a predetermined time along a first path according to a test situation from among a plurality of base scenes;
generating at least two visibility test simulations by matching at least two test signs with the first base scene;
acquiring gaze data of the user while reproducing the at least two visibility test simulations, respectively;
comparing visibility parameters of the at least two test signs based on each of the gaze data; and
It is set to include an operation of selecting a sign with the highest visibility among the at least two test signs,
The visibility parameter is the time it took for the user to see the test sign for the first time (Time to First Fixation), the total time the user's gaze stays on the test sign (Total Duration of Fixations), and the gaze data stop in the entire simulation. At least one of entropy (Scene Stationary Gaze Entropy), stop entropy excluding the time the gaze stayed on the sign in the entire simulation (Target Stationary Gaze Entropy), and transition entropy in the entire simulation (Scene Transition Gaze Entropy). Recording medium.

Images