JP2021051565A - Display device, display method, and display program - Google Patents

Abstract

To provide a display device, a display method, and a display program capable of reducing the burden on a user.SOLUTION: A display device 100 includes: a main body 10 having a display 11; a video acquisition unit 31 configured to acquire a video to be displayed on the display 11; a video analysis unit 32 configured to extract a predetermined object from the acquired video; a preference storage unit 36a configured to store preference data indicating preferences for the predetermined object; a calculation unit 33 configured to obtain a preference coincidence degree indicating the degree of coincidence between the preference data stored in the preference storage unit 36a and the object extracted by the video analysis unit 32; and a display control unit 35 configured to display the predetermined object on the display in a predetermined display mode based on the preference coincidence degree.SELECTED DRAWING: Figure 1

Description

The present invention relates to a display device, a display method, and a display program.

The information distribution system that distributes video so that the receiver in a remote location can feel the presence by making it possible to communicate between the distribution device on the distributor side and the reception device on the receiver side is known. (See, for example, Patent Document 1). In such an information distribution system, a receiver in a remote location is equipped or held by a distributor as a distribution device, for example, a display device having a display unit such as a head-mounted display (hereinafter referred to as HMD) or a smartphone. It is possible to do shopping on your behalf.

Japanese Unexamined Patent Publication No. 2009-259135

In the above technology, the user (distributor) of the display device, which is a shopping agent, provides the receiver with a video of the product according to the preference of the requester (receiver), which is in line with the recipient's intention. It is required to act like this. Therefore, the user needs to communicate frequently with the client, which may increase the burden.

The present invention has been made in view of the above, and an object of the present invention is to provide a display device, a display method, and a display program capable of reducing the burden on the user.

The display device according to the present invention relates to a main body unit having a display unit, an image acquisition unit that acquires an image to be displayed on the display unit, an image analysis unit that extracts a predetermined object from the acquired image, and the object. The preference storage unit that stores the preference data of the object, the calculation unit that obtains the preference matching degree of the object extracted by the video analysis unit based on the preference data stored in the preference storage unit, and the preference matching degree. Based on this, a display control unit that displays the image of the object on the display unit in a predetermined display mode is provided.

The display method according to the present invention is based on acquiring an image to be displayed on a display unit provided in the main body unit, extracting a predetermined object from the acquired image, and preference data stored in advance for the object. The present invention includes obtaining the degree of preference matching of the extracted object, and displaying the image of the object on the display unit in a predetermined display mode based on the degree of preference matching.

The display program according to the present invention includes a process of acquiring an image to be displayed on a display unit provided in a main body unit, a process of extracting a predetermined object from the acquired image, and preference data indicating a preference for the predetermined object. , A process of obtaining a preference match degree indicating the degree of matching with the extracted object, and a process of displaying the predetermined object on the display unit in a predetermined display mode based on the preference match degree are executed on the computer. Let me.

According to the present invention, it is possible to reduce the burden on the distributor.

FIG. 1 is a diagram showing an example of a display device according to the present embodiment. FIG. 2 is a diagram showing an example of preference data stored in the preference storage unit. FIG. 3 is a diagram showing an example of environmental information stored in the environmental information storage unit. FIG. 4 is a diagram showing an example of display contents in the display unit of the display device. FIG. 5 is a diagram showing another example of display contents in the display unit of the display device. FIG. 6 is a flowchart showing an example of the display method according to the present embodiment.

Hereinafter, a display device, a display method, and an embodiment of a display program according to the present invention will be described with reference to the drawings. The present invention is not limited to this embodiment. In addition, the components in the following embodiments include those that can be easily replaced by those skilled in the art, or those that are substantially the same.

FIG. 1 is a diagram showing an example of a display device 100 according to the present embodiment. As shown in FIG. 1, the display device 100 includes a main body unit 10, a photographing unit 20, and a control device 30. The main body 10 has a mounting mechanism that can be worn by the user on the head. The main body 10 has a display 11. The display unit 11 can display information such as an image. In this embodiment, a configuration in which a head-mounted display (HMD) device is used as an example of the display device 100 will be described. In this case, the display unit 11 is a head-mounted display such as a so-called VR (Virtual Reality) device or AR (Augmented Reality) glass, and is arranged at a position where the user wears it on the head and covers both eyes of the user. Will be done.

The photographing unit 20 is attached to the main body 10. As the photographing unit 20, for example, a camera or the like is used. The photographing unit 20 can capture an image from the user's point of view while the user wears the main body 10 on the head. The photographing unit 20 transmits the captured image to the control device 30.

The control device 30 includes a processing device such as a CPU (Central Processing Unit) and a storage device such as a RAM (Random Access Memory) or a ROM (Read Only Memory). The control device 30 includes a video acquisition unit 31, a video analysis unit 32, a calculation unit 33, an environment information detection unit 34, a display control unit 35, a storage unit 36, and a communication unit 37.

The image acquisition unit 31 acquires the image captured by the photographing unit 20. The image analysis unit 32 extracts a predetermined object from the acquired image. Examples of the predetermined object include products sold at a predetermined store. The image analysis unit 32 can extract an object when it appears in the image by performing image processing or the like on the acquired image. The storage unit 36, which will be described later, stores data and the like for the video analysis unit 32 to perform various determinations in image processing.

An object contains one or more elements. For example, the case where the object is clothes will be described as an example. Clothes include genres (polo shirts, T-shirts, Y-shirts, trousers, etc.), shapes (overall contours, parts: collars, sleeves, buttons, pockets, etc.), dimensions (overall dimensions, parts: collars, sleeves, etc.) Dimension), color or pattern, material, etc.

When purchasing clothes, the preference for the elements of the object differs depending on the person or the time of purchase. For example, regarding the types of clothing, there are those who prefer polo shirts and those who prefer T-shirts. In addition, summer clothing is likely to be preferred in summer, and winter clothing is likely to be preferred in winter. In the present embodiment, element data indicating the preference (preference) of the purchaser (for example, the receiver) regarding the element of such an object is stored in the preference storage unit 36a of the storage unit 36 as preference data. Further, the preference data may be set separately from the element of the object. For example, preference data may be set based on the purchaser's previous purchase history. In addition, by acquiring information from the purchaser's smartphone or the like, preference data can be set based on, for example, Internet browsing history or search history.

The calculation unit 33 obtains the preference matching degree of the object extracted by the image analysis unit 32 based on the preference data stored in the preference storage unit 36a. The preference matching degree is a value indicating the preference matching degree between the extracted object and the preference shown in the preference data. When determining the degree of preference matching, the calculation unit 33 calculates a score based on the degree of similarity between the extracted object element and the element data of the object element stored in the preference storage unit 36a of the storage unit 36. Calculate and calculate the degree of preference matching based on the total score. For example, the higher the score, the higher the degree of preference matching. In addition, the lower the score, the lower the degree of preference matching. Further, for example, when the object is clothing, elements such as shape or color are important factors in selecting the object and have a great influence. Therefore, the similarity of elements such as shape or color can be weighted as compared with other elements to calculate the score. In addition, the calculation unit 33 may calculate the degree of preference agreement in two stages of whether the score is higher or lower than a predetermined threshold value.

The environmental information detection unit 34 detects environmental information about the environment around the main body 10. The environmental information detection unit 34 stores the detected environmental information in the environmental information storage unit 36b of the storage unit 36. The environmental information includes, for example, the state of the user who attaches the main body 10 (moving, stopped moving, talking, etc.), the state of the photographing unit 20 attached to the main body 10 (angle of view of the camera, etc.), and the main body. The location where 10 is located (indoor, outdoor, inside the store, outside the store, etc.), the situation inside the store when the main body 10 is located inside the store (distance to the product, product price, product inventory, etc.), weather (weather, weather, etc.) Temperature, humidity, etc.). The specific configuration of the environmental information detection unit 34 will be described. The environmental information detection unit 34 includes a GPS sensor and detects the position information to detect the environmental information indicating the state of the user (moving, stopped moving, etc.). Further, the environment information detection unit 34 detects the environment information indicating the state of the photographing unit 20 (camera angle of view, etc.) by reading out the settings such as the angle of view of the camera stored in the storage unit 36. Further, the environmental information detection unit 34 reads the map information stored in the storage unit 36 and compares the position information with the map information to determine the location where the main body 10 is located (indoor, outdoor, inside the store, outside the store, etc.). Is detected. Alternatively, when the environmental information detection unit 34 includes an antenna module capable of receiving radio waves from Wi-Fi (registered trademark) or Bluetooth (registered trademark) and receives radio waves from a wireless transmitter installed in a store or the like. Detects the location (indoor, store, etc.) where the main body 10 is located. Alternatively, the environmental information detection unit 34 acquires the image taken from the image acquisition unit 31, and when the QR code (registered trademark) indicating the store is analyzed from the image, the place where the main body unit 10 is located ( Detect indoors, in stores, etc.). Further, the environmental information detection unit 34 includes an antenna module that can be connected to the Internet, and the situation in the store (distance to the product, product price, product inventory, etc.), weather (weather, etc.) from a specific server via the Internet. Detects environmental information indicating temperature, humidity, etc.). The environmental information detection unit 34 may include all of the above-described configurations, or may include only a part of the above configuration.

The display control unit 35 controls the display operation of the display unit 11. The display control unit 35 causes the display unit 11 to display the image acquired by the image acquisition unit 31. Further, the display control unit 35 displays the image of the object on the display unit 11 in a predetermined display mode based on the preference matching degree calculated by the calculation unit 33. As a predetermined display mode, highlighting and the like of highlighting and displaying with respect to other areas can be mentioned. For example, a display mode that emphasizes the outline of an object, a display mode that displays a bounding box for an object, a display mode that enlarges and displays an object, a display mode that displays effects (glitter, aura, etc.) for an object, and other than an object. A display mode that reduces the visibility of the area of the above, and the like can be mentioned.

The storage unit 36 stores various information. The storage unit 36 has storage such as a hard disk drive or a solid state drive. An external storage medium such as a removable disk may be used as the storage unit 36. The storage unit 36 may be provided independently of the control device 30. The storage unit 36 has a preference storage unit 36a and an environmental information storage unit 36b as described above.

The preference storage unit 36a stores preference data for the elements of the object. FIG. 2 is a diagram showing an example of preference data stored in the preference storage unit 36a. As shown in FIG. 2, in the preference storage unit 36a, the elements (genre, shape, color, etc.) of the object, the specific type of iodine, and the coefficient for the specific type are stored in association with each other. The coefficient is a weighting value when calculating the score when determining the degree of preference matching. The coefficient is set to a value for each target person according to the preference of the target person.

The environmental information storage unit 36b stores the environmental information detected by the environmental information detection unit 34. FIG. 3 is a diagram showing an example of environmental information stored in the environmental information storage unit 36b. As shown in FIG. 3, the environmental information storage unit 36b stores a specific type of detected environmental information and environmental information about the type in association with each other.

Further, the storage unit 36 includes a process of acquiring an image to be displayed on the display unit 11 provided in the main body unit 10, a process of extracting a predetermined object from the acquired image, and preference data indicating a preference for the predetermined object. A display program that causes a computer to execute a process of obtaining a preference match degree indicating the degree of matching with an extracted object and a process of displaying a predetermined object on the display unit 11 in a predetermined display mode based on the preference match degree. Remember.

The communication unit 37 can communicate information via a network (not shown). The communication unit 37 may perform either wired or wireless communication. The communication unit 37 can transmit, for example, as information, a video acquired by the video acquisition unit 31, a video of an object displayed by the display control unit 35 in a predetermined display mode, and the like.

In the present embodiment, a case where the user acts as an agent for shopping for clothes (objects) at an actual store at the request of a receiver (requester) in a remote location will be described as an example. On the receiver side, a receiving device 200 capable of receiving information transmitted from the communication unit 37 of the display device 100 is provided. The receiving device 200 has a display unit (not shown). In the receiving device 200, the video transmitted from the communication unit 37 is displayed on the display unit 201. By looking at the display unit 201 of the receiving device 200, the receiver, like the user, displays the image displayed on the display unit 11 of the display device 100, that is, the image acquired by the photographing unit 20 and a predetermined display mode. You can see the image of the object displayed in.

Next, the operation of the display device 100 configured as described above will be described. The user attaches the main body 10 of the display device 100 to the head and goes to a store that sells the object to be purchased. The photographing unit 20 captures an image from the user's perspective. The display control unit 35 causes the display unit 11 to display the image acquired by the image acquisition unit 31. The environmental information detection unit 34 acquires the environmental information around the main body 10 and stores the environmental information in the environmental information storage unit 36b of the storage unit 36. The recipient's preference data for the object is stored in advance in the preference storage unit 36a of the storage unit 36.

FIG. 4 is a diagram showing an example of display contents on the display unit 11 of the display device 100. When the environmental information detection unit 34 detects that the user is moving outdoors or indoors, the display control unit 35 displays an image of an object to be purchased that includes an element preferred by the recipient. It can be displayed on the display unit 11. As shown in FIG. 4, the display control unit 35 can show an image of an object in a predetermined display area 12 of the display unit 11, for example. Note that in FIG. 4, images other than the image of the object are omitted.

At this time, the display control unit 35 can adjust the number of objects to be displayed on the display unit 11 according to the environmental information. For example, the display control unit 35 increases the number of displays when the angle of view of the photographing unit 20 is large, decreases the number of displays when the angle of view is small, and so on, depending on the setting of the angle of view of the photographing unit 20. You can adjust the number of objects displayed. Further, for example, the display control unit 35 aims to reduce the number of objects displayed when the main body 10 is located outdoors and increase the number of objects displayed when the main body 10 is located indoors. If the object is located in a store that sells the object, the number of displayed objects can be further increased. Further, the display control unit 35 may display a smaller number of objects when the weather is bad (rain, snow, strong wind, etc.) than when the weather is good (sunny, cloudy, etc.).

When the user enters the target store, the environmental information detection unit 34 changes the environmental information around the main body 10 from outside the store to inside the store. When the environmental information is changed in the store, the display control unit 35 changes the display mode of the object. FIG. 5 is a diagram showing another example of the display content of the display unit 11. While the user is in the store, the display control unit 35 expands the display area 12 of the object preferred by the receiver, as shown in FIG.

Further, the image acquisition unit 31 acquires the image captured by the photographing unit 20. The image analysis unit 32 extracts clothing objects from the acquired video. The calculation unit 33 obtains the preference matching degree of the object extracted by the image analysis unit 32 based on the preference data stored in the preference storage unit 36a. For example, the calculation unit 33 calculates the score of the extracted object, determines that the preference matching degree is high when it is higher than the predetermined threshold value, and determines that the preference matching degree is low when it is lower than the predetermined threshold value. To do. Further, the environmental information detection unit 34 detects the environmental information about the environment around the main body 10 and stores the detected environmental information in the environmental information storage unit 36b of the storage unit 36.

The display control unit 35 displays the image of the object on the display unit 11 in a predetermined display mode based on the preference matching degree calculated by the calculation unit 33 and the environmental information detected by the environmental information detection unit 34. FIG. 5 is a diagram showing another example of the display content on the display unit 11 of the display device 100. As shown in FIG. 5, the display control unit 35 can display the objects P1, P2, and P3 extracted by the image analysis unit 32 on the display unit 11, respectively. Of these, for example, the object P1 determined to have a high degree of matching may be enlarged and displayed as compared with the other objects P2 and P3. Further, the display control unit 35 may display, for example, the object P1 having the highest degree of matching in the center of the display area 13. Further, the display control unit 35 may have a display mode in which the object P3 determined to have a low degree of coincidence is, for example, shaded as a whole to reduce the visibility. The predetermined display form of the object is not limited to the above, and may be another display form.

Further, the display control unit 35 sets the distance between the objects P1 and P2 determined to have a high degree of matching in the store, for example, when the distance between the product corresponding to the objects P1 and P2 and the main body 10 is close. It may be a display mode such as increasing the emphasis of the display as it approaches. For example, the display control unit 35 analyzes an image in which a product corresponding to the objects P1 and P2 is photographed, and based on a change in the size of the photographed product in several tens of seconds, the product and the main body unit 10 It may be determined whether or not the distance is approaching.

Further, the environmental information detection unit 34 may acquire product information about at least one of the price and inventory of the product corresponding to the objects P1 and P2 from the store, for example. In this case, the display control unit 35 can highlight the objects P1 and P2 according to the product information. The display control unit 35 may highlight the objects P1 and P2, for example, when the price of the product corresponding to the objects P1 and P2 is a discounted price. Further, when the inventory of the products corresponding to the objects P1 and P2 is low, highlighting such as blinking the objects P1 and P2 may be performed so that it can be determined that the inventory is low. The highlighting mode is not limited to the above mode, and may be other modes.

Further, when the display control unit 35 displays the objects on the display unit 11, for example, when the main body unit 10 is located indoors, the number of objects to be displayed is reduced as compared with the case where the main body unit 10 is not located indoors. You can do a lot. In this way, the display control unit 35 can adjust the number of objects to be displayed according to the environmental information.

FIG. 6 is a flowchart showing an example of the display method according to the present embodiment. As shown in FIG. 6, the image acquisition unit 31 acquires the image captured by the photographing unit 20 (step S10). The display control unit 35 displays the acquired video on the display unit 11. Further, the image analysis unit 32 extracts a predetermined object from the acquired image (step S20). In step S20, the image analysis unit 32 can extract a predetermined object by performing image processing such as pattern matching with a template prepared for each predetermined object. The method for extracting a predetermined object by the video analysis unit 32 is not limited to this, and various image processing methods can be used.

The calculation unit 33 obtains the preference matching degree of the extracted object based on the preference data stored in the preference storage unit 36a of the storage unit 36 in advance for the object (step S30). In step S30, the calculation unit 33 calculates a score according to the degree of similarity with the element data for each element of the object, and obtains the degree of preference matching based on the total score.

The display control unit 35 displays the image of the object on the display unit 11 in a predetermined display mode based on the obtained preference match degree (step S40). In step S40, the display control unit 35 may set the display mode of the image of the object based on the environmental information acquired by the environmental information detection unit 34 in addition to the preference matching degree.

As described above, the display device 100 according to the present embodiment extracts a predetermined object from the main body unit 10 having the display unit 11, the image acquisition unit 31 for acquiring the image to be displayed on the display unit 11, and the acquired image. The degree of coincidence between the image analysis unit 32, the preference storage unit 36a that stores the preference data indicating the preference for a predetermined object, the preference data stored in the preference storage unit 36a, and the object extracted by the image analysis unit 32. A calculation unit 33 for obtaining the degree of preference matching to be shown, and a display control unit 35 for displaying a predetermined object on the display unit 11 in a predetermined display mode based on the degree of preference matching are provided.

Further, the display method according to the present embodiment is to acquire an image to be displayed on the display unit 11 provided in the main body unit 10, extract a predetermined object from the acquired image, and indicate a preference for the predetermined object. It includes obtaining a preference matching degree indicating the degree of matching between the preference data and the extracted object, and displaying a predetermined object on the display unit 11 in a predetermined display mode based on the preference matching degree.

Further, the display program according to the present embodiment shows a process of acquiring an image to be displayed on the display unit 11 provided in the main body unit 10, a process of extracting a predetermined object from the acquired image, and a preference for a predetermined object. The computer executes a process of obtaining a preference match degree indicating the degree of matching between the preference data and the extracted object, and a process of displaying a predetermined object on the display unit 11 in a predetermined display mode based on the preference match degree. Let me.

According to the present embodiment, the preference matching degree of the extracted object is obtained based on the preference data stored in advance, and the image of the object is displayed on the display unit 11 in a predetermined display mode based on the obtained preference matching degree. To do. As a result, the user can easily grasp the object to be selected without having to communicate frequently with the outside, so that the burden on the user can be reduced.

The display device 100 according to the present embodiment further includes an environment information detection unit 34 that detects environmental information about the environment around the main body 10, and the display control unit 35 displays based on the degree of preference matching and the environment information. Set the aspect. As a result, the display mode can be flexibly set according to the surrounding environment of the main body 10.

In the display device 100 according to the present embodiment, the environmental information includes information about whether or not the main body 10 is located indoors, and the display control unit 35 is the main body 10 when the main body 10 is located indoors. Display more objects than if is not located indoors. As a result, the number of objects to be displayed is reduced when the object is not located indoors, that is, when it is located outdoors, so that the user's field of view can be secured.

In the display device 100 according to the present embodiment, the object is a product sold at a predetermined store, and the environmental information includes information on whether or not the main body 10 is located in the store. As a result, the display mode can be set differently depending on whether the main body 10 is located in the store or not in the store.

In the display device 100 according to the present embodiment, the environmental information includes the distance between the main body unit 10 and the product corresponding to the object, and the display control unit 35 enlarges and displays the object as the distance approaches. As a result, the user can intuitively recognize that the product corresponding to the object is approaching.

In the display device 100 according to the present embodiment, the environmental information includes product information about at least one of the price of the object and the number of stocks of the object, and the display control unit 35 sets the display mode according to the product information. As a result, the user can be made to recognize the product information as well.

In the display device 100 according to the present embodiment, the preference data includes element data that can be set for each element of the object, and the calculation unit 33 is stored in the element of the object extracted by the video analysis unit 32 and the preference storage unit 36a. The score is calculated based on the degree of similarity with the element data, and the degree of preference matching is obtained based on the total score. As a result, the degree of preference matching can be obtained with high accuracy.

The technical scope of the present invention is not limited to the above-described embodiment, and modifications can be made as appropriate without departing from the spirit of the present invention. For example, in the above embodiment, clothing has been described as an example as an object, but the present invention is not limited to this, and other types of products may be objects. Further, the display device 100 is not limited to the HMD device of the above embodiment, and may be, for example, a terminal device having a display unit such as a smartphone.

P1, P2, P3 ... Object, 10 ... Main body unit, 11,201 ... Display unit, 12, 13 ... Display area, 20 ... Shooting unit, 30 ... Control device, 31 ... Image acquisition unit, 32 ... Image analysis unit, 33 ... Calculation unit, 34 ... Environmental information detection unit, 35 ... Display control unit, 36 ... Storage unit, 36a ... Preference storage unit, 36b ... Environmental information storage unit, 37 ... Communication unit, 100 ... Display device, 200 ... Receiver device

Claims (6)

The main body with the display and
An image acquisition unit that acquires an image to be displayed on the display unit,
An image analysis unit that extracts a predetermined object from the acquired image, and
A preference storage unit that stores preference data indicating preferences for the predetermined object, and
A calculation unit for obtaining a preference matching degree indicating a matching degree between the preference data stored in the preference storage unit and an object extracted by the video analysis unit.
A display device including a display control unit that displays the predetermined object on the display unit in a predetermined display mode based on the preference matching degree. It is further equipped with an environmental information processing unit that detects environmental information about the environment around the main unit.
The display device according to claim 1, wherein the display control unit sets the display mode based on the preference matching degree and the environmental information. The environmental information includes information on whether or not the main body is located indoors.
The display device according to claim 2, wherein the display control unit increases the number of the predetermined objects to be displayed when the main body is located indoors, as compared with the case where the main body is not located indoors. The environmental information includes the distance between the main body and the product corresponding to the predetermined object.
The display device according to claim 2 or 3, wherein the display control unit enlarges and displays the object as the distance approaches. Acquiring the image to be displayed on the display unit provided on the main unit, and
Extracting a predetermined object from the acquired video and
Obtaining the preference matching degree indicating the degree of matching between the preference data indicating the preference for the predetermined object and the extracted object, and
A display method including displaying the predetermined object on the display unit in a predetermined display mode based on the preference matching degree. The process of acquiring the image to be displayed on the display unit provided on the main unit, and
The process of extracting a predetermined object from the acquired video and
Preference data indicating the preference for the predetermined object and the process of obtaining the preference matching degree indicating the degree of matching with the extracted object.
A display program that causes a computer to execute a process of displaying the predetermined object on the display unit in a predetermined display mode based on the preference matching degree.

Images