JP6959197B2 - Garbage separation support system, terminal device, garbage separation support method, and program - Google Patents

Description

Embodiments of the present invention relate to a garbage separation support system, a terminal device, a garbage separation support method, and a program.

There is known a technique for specifying the waste separation information to be discarded according to the separation information in the local government that is trying to dispose of the waste, and also specifying the container for collecting the waste (see, for example, Patent Document 1). In this technology, the terminal sends information that identifies the local government to the server, and from the list of local government information existing on the server, the list of garbage separation information by local government, and the list of collection cooperating store information, the local government garbage of the corresponding local government. By acquiring the separation information and collating the acquired municipal waste separation information with the product separation information of the product to be disposed of, the garbage separation method can be easily specified. Further, in this technology, the container for collecting the corresponding garbage is specified from the list of the collection container garbage separation information held in the terminal, and the load of the separation work is reduced and the separation accuracy is improved.

Japanese Unexamined Patent Publication No. 2005-173961

In the conventional technology, when the user operates the terminal and inputs the attribute information of the garbage to be discarded, the information of the garbage can to be discarded can be obtained. However, unless the terminal is operated and the attribute information of the garbage is input when the garbage is thrown away, the information of the trash can to be discarded cannot be obtained. In addition, when disposing of PET bottles, one product is separated into multiple sorting targets, such as plastic containers and packaging for labels, resource waste for the main body, and plastic containers and packaging for caps, and the disposal destinations for the separated sorting targets. The trash can may break up. Therefore, it may be complicated to explain the method of separating the waste to be discarded with sentences and illustrations.
The present invention has been made to solve the above problems, and an object of the present invention is to provide a garbage separation support system, a terminal device, a garbage separation support method, and a program capable of improving the convenience of garbage separation.

(1) One aspect of the present invention is a garbage separation support system including a terminal device and a server, in which the terminal device positions an image input unit for acquiring image information of dust and the own terminal device. A signal that includes the positioning unit, the image information of the dust acquired by the image input unit, and the position information of the own terminal device positioned by the positioning unit, and requests a determination of separation of the dust. A control unit that creates a certain garbage separation determination request, and a communication unit that transmits the garbage separation determination request created by the control unit to the server and receives a garbage separation response that is a response signal to the transmitted garbage separation determination request. And, based on the image information indicating the dust separation method included in the dust separation response received by the communication unit and the image information indicating the acquired dust separation method, the image showing the dust separation method and the image. The server includes an image output unit that synthesizes and displays the image of the garbage box obtained from the image information of the garbage box acquired by the image input unit, and the server responds to the garbage separation determination request transmitted by the terminal device. The image information of the included dust and the position information of the own terminal device are acquired, and the feature amount of the image of the dust is extracted based on the acquired image information of the dust, and the extracted dust is extracted. The garbage separation name corresponding to the feature amount of the image is acquired from a database stored in association with the feature amount of the garbage image and the garbage separation name shown in the garbage image, and the acquired garbage Based on the sorting name and the position information of the own terminal device, information indicating the dust sorting method is acquired, and based on the acquired information indicating the dust sorting method, the dust sorting method. An image recognition unit that acquires image information indicating the above and creates the dust separation response including the image information indicating the acquisition method of the dust, and the terminal device that creates the dust separation response created by the image recognition unit. The image input unit acquires the image information of the garbage box, and the control unit creates the garbage separation determination request including the image information of the garbage box. The image recognition unit further acquires the image information of the garbage box included in the garbage separation determination request transmitted by the terminal device, and obtains the image information of the garbage box and the position information of the own terminal device. Based on this, the position information of the garbage box is acquired, and based on the acquired sorting name of the garbage and the position information of the garbage box, information indicating the sorting method of the garbage is acquired. You, a garbage separation support system.
( 2 ) One aspect of the present invention is the dust separation support system according to (1) above, wherein the terminal device includes an attitude detection unit that detects the orientation of the own terminal device, and the image output unit is the image output unit. Based on the information indicating the orientation detected by the posture detection unit, the position of the image indicating the method of separating the dust to be combined with the image obtained from the image information of the dust box acquired by the image input unit is corrected. ..
( 3 ) One aspect of the present invention is the dust separation support system according to the above (1) or ( 2) , wherein the image recognition unit uses the acquired image information of the dust and the position of the own terminal device. Based on the information, the position information of the trash can in a predetermined range is acquired from the position of the own terminal device, and the server communication unit further includes the position information of the trash can acquired by the image recognition unit. To create.
( 4 ) One aspect of the present invention includes an image input unit that acquires image information of dust and image information of a dust box, a positioning unit that positions the position of the own terminal device, and the dust acquired by the image input unit. Based on the image information of the above, the feature amount of the dust image is extracted, and the sorting name of the dust corresponding to the feature amount of the extracted dust image is given to the feature amount of the dust image and the image of the dust. Based on the separated name of the garbage, the image information of the garbage box, and the position information of the own terminal device positioned by the positioning unit, which is acquired from the database stored in association with the garbage sorting name indicated by. Based on the acquired position information of the garbage box, information indicating the method for separating the garbage is acquired, and based on the acquired information indicating the method for separating the garbage, image information indicating the method for separating the garbage is obtained. Based on the image recognition unit that acquires the above and the image information that indicates the separation method of the dust acquired by the image recognition unit, the image showing the separation method of the dust and the garbage box acquired by the image input unit. It is a terminal device including an image output unit that synthesizes and displays the image of the garbage box obtained from the image information.
( 5 ) One aspect of the present invention is a garbage separation support method executed by a garbage separation support system including a terminal device and a server, and the terminal device acquires image information of garbage and image information of a garbage box. Steps to be performed, a step in which the terminal device determines the position of the own terminal device, and the terminal device obtains the image information of the garbage, the image information of the garbage box, and the position information of the own terminal device. A step of creating a garbage separation determination request, which is a signal requesting the determination of garbage separation, a step of the terminal device transmitting the garbage separation determination request to the server, and a step of the server sending the garbage separation determination request to the server. The step of acquiring the image information of the garbage, the image information of the garbage box, and the position information of the own terminal device included in the garbage separation determination request transmitted by the server, and the server of the garbage. Based on the image information, the feature amount of the dust image is extracted, and the classification name of the dust corresponding to the feature amount of the extracted dust image is shown by the feature amount of the dust image and the dust image. obtaining from a database that stores in association with the sorting names of garbage to be the server, and fractionation name of the waste, it is obtained on the basis of the said positional information of the image information and the own terminal device of the trash The step of acquiring the information indicating the garbage sorting method based on the position information of the garbage box, and the server showing the garbage sorting method based on the information indicating the garbage sorting method. A step of acquiring image information, a step of the server creating a garbage separation response including the image information indicating the method of separating the garbage, and a step of the server transmitting the garbage separation response to the terminal device. The terminal device acquires the step of receiving the dust separation response transmitted by the server, and the terminal device acquires the image information indicating the method of separating the dust included in the dust separation response. A step of synthesizing and displaying an image showing the method of separating the garbage and the image of the garbage obtained from the acquired image information of the garbage box based on the image information indicating the acquired method of separating the garbage. It is a garbage separation support method that has.
( 6 ) One aspect of the present invention is the image information of the garbage included in the garbage separation determination request transmitted by the terminal device, the image information of the garbage box, and the terminal device to the computer of the server communicating with the terminal device. Based on the step of acquiring the position information of the dust and the image information of the acquired dust, the feature amount of the dust image is extracted, and the dust corresponding to the feature amount of the extracted dust image is separated. A step of acquiring a name from a database that stores a feature amount of a garbage image and a garbage sorting name indicated by the garbage image in association with each other, the obtained sorting name of the garbage, and the image information of the garbage box. And the step of acquiring the information indicating the garbage sorting method based on the position information of the garbage box acquired based on the position information of the terminal device and the information indicating the acquired garbage sorting method. Based on the above, a step of acquiring image information indicating the method for separating the garbage and a garbage separation response including the image information indicating the method for separating the garbage are created, and the created garbage separation response is transmitted to the terminal. It is a program that executes a step of transmitting to a device.

According to the present invention, it is possible to provide a garbage separation support system, a terminal device, a garbage separation support method, and a program that can improve the convenience of garbage separation.

It is a figure which shows an example of the garbage separation support system of embodiment of this invention. It is a figure which shows an example of the hardware composition of the server which comprises the garbage separation support system of embodiment of this invention, and the hardware composition of a terminal apparatus. It is a block diagram which shows the server which comprises the garbage separation support system of embodiment of this invention, and the terminal apparatus. It is a sequence chart which shows an example of the operation of the garbage separation support system of embodiment of this invention. It is a figure which shows an example of the operation of the garbage separation support system of embodiment of this invention. It is a figure which shows an example of the operation of the garbage separation support system of embodiment of this invention. It is a figure which shows an example of the operation of the garbage separation support system of embodiment of this invention. It is a figure which shows an example of the operation of the garbage separation support system of embodiment of this invention. It is a figure which shows an example of the operation of the garbage separation support system of embodiment of this invention. It is a block diagram which shows the terminal apparatus which comprises the garbage separation support system of embodiment of this invention.

Next, the garbage separation support system, the terminal device, the garbage separation support method, and the program of the present embodiment will be described with reference to the drawings. The embodiments described below are merely examples, and the embodiments to which the present invention is applied are not limited to the following embodiments. In all the drawings for explaining the embodiment, the same reference numerals are used for those having the same function, and the repeated description will be omitted.
Further, "based on XX" in the present application means "based on at least XX", and includes a case where it is based on another element in addition to XX. Further, "based on XX" is not limited to the case where XX is used directly, but also includes the case where XX is calculated or processed. "XX" is an arbitrary element (for example, arbitrary information).

(Embodiment)
(Garbage separation support system)
As shown in FIG. 1, the garbage separation support system 3 of the present embodiment includes a server 1 and a terminal device 2. The server 1 and the terminal device 2 are connected via a network 4 such as a mobile phone network or the Internet. Trash cans 5 are installed in public facilities such as tourist spots and airports where many foreigners live. Tourists such as visitors to Japan activate the garbage separation application installed in the terminal device 2. When the garbage separation application is activated, the terminal device 2 displays a message prompting the user to take a picture (imaging) of the garbage and the trash can, such as "Please take a picture of the garbage and the trash can to be separated".
The user (tourist) of the terminal device 2 photographs the garbage and the trash can according to the message displayed on the terminal device 2. When the user takes a picture, the terminal device 2 generates the image information of the garbage and the image information of the garbage box. The terminal device 2 creates a garbage separation determination request including the identification information of the terminal device 2, the position information of the terminal device 2, the generated image information of the garbage, and the image information of the trash can, and the created garbage separation determination request. Is transmitted to the server 1.

The server 1 identifies attributes such as the garbage separation name based on the garbage image information included in the garbage separation determination request transmitted by the terminal device 2, and identifies the configuration of the trash can based on the image information of the trash can. , Based on the identification result of the garbage attribute and the identification result of the composition of the trash can, the explanation of the garbage separation method and the information for explaining the garbage separation method (hereinafter referred to as "garbage separation method explanation information") are provided. get. The method of separating garbage follows the rules of the area where the trash can 5 is installed. The server 1 includes the acquired garbage separation method explanatory information, creates a garbage separation response destined for the terminal device 2, and transmits the created garbage separation response to the terminal device 2. The terminal device 2 receives the garbage separation response transmitted by the server 1, and is provided with the terminal device 2 by augmented reality technology (AR) based on the garbage separation method explanatory information included in the received garbage separation response. The dust separation method is combined and displayed with the image of the dust bin taken by the image pickup unit.
In the example shown in FIG. 1, the trash can 5 includes three separate trash cans, a combustible trash can 51, a non-combustible trash can 52, and a resource trash can 53. Identification information such as different colors and characters is assigned to each of the plurality of separated trash cans such as the combustible trash can 51, the non-combustible trash can 52, and the resource trash can 53. For example, when different colors are assigned to each of the plurality of sorting trash cans, as an example, the housing of the combustible trash can 51 is painted in red, the housing of the non-combustible trash can 52 is painted in blue, and resources are used. The housing of the trash can 53 is painted green.
When the trash can 5 is installed, the installer provides information such as image information and text information of the trash can 5 that can identify the characteristics of the trash can and information indicating the configuration of the trash can 5 (hereinafter referred to as "trash can configuration information"). Input to server 1. Hereinafter, the case where the image information of the trash can 5 is used as an example of the information that can determine the characteristics of the trash can will be described. The server 1 derives the feature amount of the image of the trash can 5 based on the image information of the trash can 5, and stores the feature amount of the derived image of the trash can 5 and the trash box configuration information in association with each other. Specifically, the installer inputs the image information of the trash can 5 by operating the server 1 and causes the installer to recognize the feature amount of the input image of the trash can 5. The installer inputs the trash can configuration information, and stores the feature amount of the image of the trash can 5 and the trash can configuration information in association with each other in the server 1. An example of the trash can configuration information is information indicating whether each separated trash can corresponds to any of combustible waste, non-combustible waste, and recyclable waste. Here, combustible garbage is combustible garbage, and specific examples thereof include raw garbage, paper waste, fiber waste, plastics, rubber and leather. Non-burnable garbage is non-burnable garbage, and specific examples thereof include products made of metal, glass, and ceramics. Recyclable waste is generally recyclable waste, and mainly includes cans, PET bottles, bottles, papers, batteries, and metal ingots. The server 1 stores the feature amount of the image of the trash can 5 and the trash can configuration information in association with each other.

The case where the garbage separation support system is actually used will be described. A person who intends to dispose of garbage (hereinafter referred to as "user") activates a garbage separation application by operating the terminal device 2. When the garbage separation application is activated, the terminal device 2 displays a message prompting the photographer to take a picture of the garbage and the trash can. The user of the terminal device 2 photographs the garbage and the trash can according to the message displayed on the terminal device 2. When the user takes a picture, the terminal device 2 generates the image information of the garbage and the image information of the garbage box. The terminal device 2 requests a garbage separation determination including the generated garbage image information, the garbage box image information, the information indicating the orientation of the terminal device 2, the identification information of the terminal device 2, and the position information of the terminal device 2. Is created, and the created garbage separation determination request is transmitted to the server 1. The server 1 receives the garbage separation determination request transmitted by the terminal device 2, and receives the garbage image information included in the received garbage separation determination request, the image information of the trash can, the information indicating the orientation of the terminal device 2, and the terminal. The identification information of the device 2 and the position information of the terminal device 2 are acquired. The server 1 extracts the feature amount of the dust image from the dust image based on the acquired dust image information. The server 1 extracts the feature amount of the image of the trash can from the image of the trash can based on the acquired image information of the trash can.
The server 1 stores the garbage image from a database that stores the feature amount of the garbage image and the garbage classification name shown by the garbage image in association with each other based on the information indicating the feature amount of the extracted garbage image. Search for the separated name of garbage that corresponds to the feature amount of. Further, the server 1 converts the feature amount of the image of the trash can into the feature amount of the image of the trash can from the database stored in association with the feature amount of the image of the trash can and the configuration information of the trash can based on the information indicating the feature amount of the extracted image of the trash can. Acquire information indicating the configuration information of the corresponding trash can. Further, the server 1 estimates the position of the trash can 5 based on the acquired position information of the terminal device 2 and the information indicating the orientation of the terminal device. Based on the estimation result of the position of the trash can 5, the server 1 stores the information indicating the area in association with the method of separating the garbage in the area, and the area corresponding to the estimation result of the position of the trash can 5 from the database. Acquires the information indicating the garbage separation method stored in association with the information indicating.
Further, the server 1 displays information indicating the garbage separation method from a database that stores information indicating the garbage separation method in association with the information indicating the garbage separation method based on the acquired information indicating the garbage separation method. Obtain information on how to separate garbage that corresponds to. The server 1 creates a garbage separation response including the acquired information on how to separate the garbage, and transmits the created garbage separation response to the terminal device 2.
The terminal device 2 receives the garbage separation response transmitted by the server 1, and causes the user of the terminal device 2 to take an image of the trash can 5 based on the received garbage separation response. The terminal device 2 synthesizes and displays the garbage separation method explanatory information in the trash can 5 imaged by the terminal device 2 by the augmented reality technology based on the garbage separation method explanatory information included in the garbage separation response. Hereinafter, the server 1 and the terminal device 2 that constitute the garbage separation support system will be described.

(Hardware configuration of garbage separation support system)
With reference to FIG. 2, the hardware configuration of the server 1 and the hardware configuration of the terminal device 2 constituting the garbage separation support system 3 will be sequentially described. The server 1 includes a CPU 152, a memory 154, a non-volatile memory 156, an operation unit 162, a communication I / F 166, and an internal bus 150. The CPU 152 executes, for example, a program 1562 stored in the non-volatile memory 156, and uses the memory 154 as a work memory to control each part of the server 1. The memory 154 is composed of a RAM such as a volatile memory using a semiconductor element, and is used as a work memory of the CPU 152. The non-volatile memory 156 is composed of, for example, a ROM or the like. The non-volatile memory 156 stores a program 1562 executed by the CPU 152, terminal information 1563, garbage-specific feature amount 1564, garbage box-specific feature amount 1565, area-specific sorting information 1566, and sorting method explanation information 1567. Will be done. The terminal information 1563, the feature amount 1564 for each garbage, the feature amount 1565 for each trash can, the sorting information 1566 for each area, and the sorting method explanatory information 1567 will be described later.
The operation unit 162 is an input device that accepts operations by the installer of the trash can 5. Specifically, the installer inputs the image information of the trash can 5 and the trash can configuration information by operating the operation unit 162. The communication I / F 166 is an interface for transmitting and receiving data by communicating with an external device, a terminal device 2 connected to the network 4, and the like. Specifically, the communication I / F 166 is composed of a wireless device that performs wireless communication by wireless communication technology such as Wi-Fi (registered trademark). Further, the communication I / F 166 may be configured by a device that communicates by a communication technology such as a wired LAN. The internal bus 150 connects the CPU 152, the memory 154, the non-volatile memory 156, the operation unit 162, and the communication I / F 166 to each other. Each part connected to the internal bus 150 can exchange data with each other via the internal bus 150.

The terminal device 2 includes a CPU 252, a memory 254, a positioning unit 255, a non-volatile memory 256, an operation unit 257, a display unit 258, an imaging unit 259, a sensor 260, a communication I / F 266, and an internal bus. It has 250 and. The CPU 252 executes, for example, a program 2562 stored in the non-volatile memory 256, and uses the memory 254 as a work memory to control each part of the terminal device 2. The memory 254 is composed of a RAM such as a volatile memory using a semiconductor element, and is used as a work memory of the CPU 252. The non-volatile memory 256 is composed of, for example, a ROM or the like. The non-volatile memory 256 stores a program 2562 executed by the CPU 252. The positioning unit 255 positions the terminal device 2 by using, for example, positioning using a GPS satellite or positioning using a base station of a mobile phone network. The operation unit 257 is configured by, for example, a touch panel or the like, detects a touch operation on the screen displayed on the display unit 258, and outputs the detection result of the touch operation to the CPU 252. Touch operations include operations such as launching a garbage separation application. The display unit 258 is composed of, for example, a touch panel, and displays a message urging the photograph of the dust and the trash can. The image pickup unit 259 is composed of, for example, a camera, and captures a still image and a moving image of an external landscape. The sensor 260 is composed of an acceleration sensor, a geomagnetic sensor, and the like, and detects the orientation of the terminal device 2.
The communication I / F 266 is an interface for transmitting and receiving data by communicating with an external device, a server 1 connected to the network 4, and the like. Specifically, the communication I / F 266 is composed of a wireless device that performs wireless communication by wireless communication technology such as Wi-Fi (registered trademark). Further, the communication I / F 266 may be configured by a device that communicates with a communication technology such as a wired LAN. The internal bus 250 connects the CPU 252, the memory 254, the positioning unit 255, the non-volatile memory 256, the operation unit 257, the display unit 258, the image pickup unit 259, the sensor 260, and the communication I / F 266 to each other. Each part connected to the internal bus 250 can exchange data with each other via the internal bus 250.

(Functional configuration of the garbage separation support system according to this embodiment)
As shown in FIG. 3, the server 1 includes a server communication unit 101, a terminal identification unit 102, a terminal information storage unit 103, an image recognition unit 104, a garbage-specific feature amount storage unit 105, and a garbage box-specific feature amount storage. It has a section 106, a regional sorting information storage section 107, and a sorting method explanation information storage section 108. Each of these parts has a function realized by operating any of the components shown in FIG. 2 by an instruction from the CPU 152 that executes the program 1562 expanded from the non-volatile memory 156 to the memory 154. Is.
Each functional configuration of the server 1 will be described in detail with reference to FIGS. 2 and 3. The server communication unit 101 of the server 1 is realized by an instruction from the CPU 152 and communication I / F 166. The server communication unit 101 transmits and receives various data after a session is established with another device such as the terminal device 2. Specifically, the server communication unit 101 receives the garbage separation determination request transmitted by the terminal device 2, and outputs the received garbage separation determination request to the terminal identification unit 102. The garbage separation determination request includes identification information of the terminal device 2, position information of the terminal device 2, information indicating the orientation of the terminal device 2, image information of the garbage, and image information of the trash can. Here, the identification information of the terminal device 2 is an IP address, a MAC address, or the like of the terminal device 2. Hereinafter, the case where the identification information of the terminal device 2 is the IP address of the terminal device 2 will be described. The server 1 and the terminal device 2 communicate with each other using the IP address of the terminal device 2. The position information of the terminal device 2 is information indicating a place of use of the terminal device 2 acquired by, for example, positioning using a GPS satellite or positioning using a base station of a mobile phone network.
The information indicating the orientation of the terminal device 2 includes information indicating the inclination of the terminal device 2 detected by the sensor 260 of the terminal device 2 and information indicating the direction. The position of the trash can 5 is estimated from the information indicating the orientation of the terminal device 2 and the position information of the terminal device 2. The dust image information is the dust image information to be sorted, which is captured by the terminal device 2. The image information of the trash can is the image information of the trash can in which the user is about to dispose of the trash. Further, the server communication unit 101 acquires the dust separation response output by the image recognition unit 104, and transmits the acquired dust separation response to the terminal device 2.

The terminal identification unit 102 is realized by an instruction from the CPU 152. The terminal identification unit 102 acquires the garbage separation determination request output by the server communication unit 101. The terminal identification unit 102 includes the identification information of the terminal device 2 included in the acquired garbage separation determination request, the position information of the terminal device 2, the information indicating the direction of the terminal device 2, the image information of the garbage, and the image of the garbage box. Get information and. The terminal identification unit 102 outputs the acquired identification information of the terminal device 2, the position information of the terminal device 2, and the information indicating the posture of the terminal device 2 to the terminal information storage unit 103. Further, the terminal identification unit 102 image-recognizes the acquired identification information of the terminal device 2, the position information of the terminal device 2, the information indicating the orientation of the terminal device 2, the image information of the garbage, and the image information of the garbage box. Output to unit 104.
The terminal information storage unit 103 is realized by the terminal information 1563 of the non-volatile memory 156. The terminal information storage unit 103 is a database, and acquires and acquires the identification information of the terminal device 2 output by the terminal identification unit 102, the position information of the terminal device 2, and the information indicating the posture of the terminal device 2, and the acquired terminal. The identification information of the device 2, the position information of the terminal device 2, and the information indicating the posture of the terminal device 2 are stored in association with each other. The information stored in the terminal information storage unit 103 is used for interconnecting the server 1 and the terminal device 2.
The image recognition unit 104 is realized by a command from the CPU 152. The image recognition unit 104 includes the identification information of the terminal device 2 output by the terminal identification unit 102, the position information of the terminal device 2, the information indicating the orientation of the terminal device 2, the image information of the garbage, and the image information of the garbage box. To get. The image recognition unit 104 uses image processing technology or the like to analyze the image of the garbage and the image of the garbage box based on the acquired image information of the garbage and the image information of the garbage box. , Extract the feature quantity.
The image recognition unit 104 acquires information indicating the separated name stored in association with the feature amount of the extracted dust image by having the dust-specific feature amount storage unit 105 search. The image recognition unit 104 causes the garbage-specific feature amount storage unit 105 to acquire detailed sorting names such as PET bottles and cans. The image recognition unit 104 acquires information indicating a sorting name corresponding to the dust of the subject of the image output by the dust-specific feature amount storage unit 105. Further, when the image recognition unit 104 does not have the feature amount of the dust image that matches the feature amount of the dust image of the subject of the image, the information indicating from the dust-specific feature amount storage unit 105 that there is no corresponding target. To get. The image recognition unit 104 causes the trash can-specific feature amount storage unit 106 to acquire the trash can configuration information that matches the feature amount of the extracted trash can image. The image recognition unit 104 acquires the configuration information of the trash can output by the feature amount storage unit 106 for each trash can. Further, when the image recognition unit 104 does not have the feature amount of the image of the trash can that matches the feature amount of the image of the trash can of the subject of the image, the information indicating that there is no corresponding target from the feature amount storage unit 106 for each trash can. To get.

The image recognition unit 104 estimates the location where the trash can 5 is installed based on the position information of the terminal device 2 and the information indicating the orientation of the terminal device 2. When the image recognition unit 104 acquires information indicating the garbage separation name, the image recognition unit 104 causes the area-specific separation information storage unit 107 to search for the acquired garbage separation name and the estimation result of the place where the trash can 5 is installed. Get the separated information for each region that is stored in association with. The image recognition unit 104 acquires the regional classification information output by the regional classification information storage unit 107. Further, when there is no regional classification information, the image recognition unit 104 acquires information indicating that there is no corresponding target from the regional classification information storage unit 107. When the image recognition unit 104 acquires the sorting information for each region, the image recognition unit 104 causes the sorting method explanation information storage unit 108 to search for the garbage separation method explanation information stored in association with the acquired sorting information for each region. Get it. The image recognition unit 104 acquires the garbage separation method explanation information output by the separation method explanation information storage unit 108. Further, when there is no garbage separation method explanation information, the image recognition unit 104 acquires information indicating that there is no corresponding target from the separation method explanation information storage unit 108.
When the image recognition unit 104 acquires the garbage separation method explanation information output by the separation method explanation information storage unit 108, the image recognition unit 104 sends a garbage separation response including the acquired garbage separation method explanation information and the identification information of the terminal device 2. The created garbage separation response is output to the server communication unit 101 via the terminal identification unit 102. Further, when the image recognition unit 104 acquires information indicating that there is no feature amount of the dust image that matches the feature amount of the dust image of the subject of the image output by the feature amount storage unit 105 for each dust, the image recognition unit 104 obtains information indicating that there is no feature amount of the dust image. A garbage separation response including information indicating that there is no separation name and identification information of the terminal device 2 is created, and the created garbage separation response is output to the server communication unit 101 via the terminal identification unit 102. Further, when the image recognition unit 104 acquires information indicating that there is no feature amount of the image of the trash can that matches the feature amount of the image of the trash can of the subject of the image output by the feature amount storage unit 106 for each trash can, the image recognition unit 104 obtains information. A garbage separation response including information indicating that there is no feature amount of the image of the trash can that matches the feature amount of the image of the trash can of the subject of the image and the identification information of the terminal device 2 is created, and the created garbage separation response is displayed. It is output to the server communication unit 101 via the terminal identification unit 102. Further, when the image recognition unit 104 acquires the information indicating that there is no regional sorting information output by the regional sorting information storage unit 107, the information indicating that there is no regional sorting information and the identification of the terminal device 2 A garbage separation response including information is created, and the created garbage separation response is output to the server communication unit 101 via the terminal identification unit 102. Further, when the image recognition unit 104 acquires the information indicating that there is no garbage separation method explanation information output by the separation method explanation information storage unit 108, the information indicating that there is no garbage separation method explanation information and the terminal device 2 A garbage separation response including the identification information of the above is created, and the created garbage separation response is output to the server communication unit 101 via the terminal identification unit 102.

The dust-specific feature amount storage unit 105 is realized by the dust-specific feature amount 1564 of the non-volatile memory 156. The garbage-specific feature amount storage unit 105 is a database that stores the garbage separation name and the feature amount of the garbage image in association with each other. The dust-specific feature amount storage unit 105 searches for which classification name the dust of the subject of the image corresponds to, based on the feature amount of the dust image output by the image recognition unit 104. If there is a sorting name corresponding to the dust of the subject of the image, the feature amount storage unit 105 for each dust outputs information indicating the sorting name to the image recognition unit 104, and outputs the feature amount of the dust of the subject of the image to the image. If there is no dust image, information indicating that there is no feature amount of the dust image is output to the image recognition unit 104.
The feature amount storage unit 106 for each trash can is realized by the feature amount 1565 for each trash can of the non-volatile memory 156. The feature amount storage unit 106 for each trash can is a database that stores the configuration information of the trash can and the feature amount of the image of the trash can in association with each other. The feature amount storage unit 106 for each trash can searches for the configuration information of the trash can of the subject of the image based on the feature amount of the image of the trash can output by the image recognition unit 104. The feature amount storage unit 106 for each trash box outputs, if there is trash box configuration information corresponding to the feature amount of the image of the trash box of the subject of the image, the information indicating the trash box configuration information to the image recognition unit 104, and outputs the image. When there is no feature amount of the image of the trash box of the subject, information indicating that there is no feature amount of the image of the trash box is output to the image recognition unit 104.
The regional classification information storage unit 107 is realized by the regional classification information 1566 of the non-volatile memory 156. The regional sorting information storage unit 107 is a database that stores the garbage sorting method for each region. Specifically, the regional classification information storage unit 107 associates information indicating the latitude range and longitude range of the region with information indicating the garbage separation name and information indicating the garbage separation method of the garbage. Remember. The regional sorting information storage unit 107 acquires the garbage sorting name output by the image recognition unit 104 and the estimation result of the place where the trash can 5 is installed, and the acquired garbage sorting name and the trash can 5 are installed. Search for information that indicates how to separate garbage based on the estimation results of the location. The regional sorting information storage unit 107 provides information indicating the garbage sorting method, if there is a garbage sorting method corresponding to the acquired garbage sorting name and the estimation result of the place where the trash can 5 is installed. If there is no garbage separation method corresponding to the acquired garbage separation name and the estimation result of the place where the trash can 5 is installed, the information is output to the image recognition unit 104 to indicate that there is no garbage separation method. , Is output to the image recognition unit 104.
With this configuration, the area-specific sorting information storage unit 107 can search for garbage separation information in the area where the trash can 5 is located, based on the estimation result of the place where the trash can 5 is installed. The sorting method explanation information storage unit 108 is realized by the sorting method explanation information 1567 of the non-volatile memory 156. The sorting method explanatory information storage unit 108 is a database that stores information indicating a garbage sorting method in association with garbage sorting method explanatory information. Here, an example of the information for explaining the garbage separation method of the garbage separation method explanatory information is an animation display. The sorting method explanatory information storage unit 108 acquires the information indicating the dust sorting method output by the image recognition unit 104, and searches for the garbage sorting method explanatory information based on the information indicating the acquired dust sorting method.

The terminal device 2 includes an image input unit 201, a control unit 202, an image recognition unit 203, a posture detection unit 204, a position measurement unit 205, a communication unit 206, and an image output unit 207. The image input unit 201 is realized by a command from the CPU 252 and an image pickup unit 259. The image input unit 201 receives one or both of the video information of the moving image obtained by the image capturing unit 259 mounted on the terminal device 2 and the image information of the still image of the moving image. Acquire either one or both of the video information and the image information of the still image. In the present embodiment, the case where the image pickup unit 259 generates the image information of the dust and the image information of the trash can by imaging the dust and the trash can will be described. The image input unit 201 outputs the acquired image information to the control unit 202.
The control unit 202 is realized by an instruction from the CPU 252. The control unit 202 controls and manages the overall operation of the terminal device 2. The control unit 202 acquires the image information of the dust output by the image input unit 201 and the image information of the trash can. Further, the control unit 202 acquires the position information of the terminal device 2 output by the position measurement unit 205. Further, the control unit 202 acquires the information indicating the direction of the terminal device 2 output by the attitude detection unit 204. The control unit 202 includes the acquired image information of the garbage, the image information of the trash can, the identification information of the terminal device 2, and the position information of the terminal device 2, and creates a garbage separation determination request destined for the server 1. , The created garbage separation determination request is output to the communication unit 206. Further, the control unit 202 acquires the garbage separation response output by the communication unit 206, and the garbage separation method explanatory information included in the acquired garbage separation response, the information indicating that there is no separation name, and the features of the image of the trash can. Obtain either information indicating that there is no amount, information indicating that there is no regional sorting information, or information indicating that there is no garbage separation method explanation information. When the control unit 202 acquires the dust separation method explanatory information, the control unit 202 processes the dust separation explanatory information by using the augmented reality technology, and obtains the image information acquired by the image input unit 201 by image processing. It is combined with the image to be created in real time. The control unit 202 outputs the image information obtained by synthesizing the dust separation method explanatory information to the image output unit 207. When the control unit 202 acquires information other than the garbage separation method explanatory information, the control unit 202 obtains the information by processing the information by using the augmented reality technology, and by performing image processing on the image information acquired by the image input unit 201. It is combined with the image to be created in real time.
The control unit 202 outputs the image information obtained by synthesizing the information other than the information for explaining the dust separation method to the image output unit 207. Specifically, when synthesizing the garbage separation method explanatory information, the control unit 202 extracts the feature amount of the image of the trash can 5 from the image of the trash can 5 acquired by the image input unit 201 to the image recognition unit 203. Let me. Further, the control unit 202 causes the posture detection unit 204 to detect the orientation of the terminal device 2. The control unit 202 separates garbage based on the feature amount of the image of the trash can 5 extracted by the image recognition unit 203 so that the garbage separation method explanatory information is displayed so as to overlap the image of the object such as the trash can 5. Method Description Adjust the position where the information is displayed. Further, the control unit 202 separates dust based on the orientation of the terminal device 2 detected by the posture detection unit 204 so that the dust separation method explanatory information is displayed so as to overlap the image of the object such as the trash can 5. Method Description Adjust the position where the information is displayed.
The image recognition unit 203 is realized by a command from the CPU 252. The image recognition unit 203 extracts the feature amount of the image from the image information acquired by the image input unit 201 by using the image processing technique. The image recognition unit 203 outputs the feature amount of the extracted image to the control unit 202 when the control unit 202 synthesizes the dust separation method explanatory information into the image in real time by using the augmented reality technology.

The posture detection unit 204 is realized by a command from the CPU 252 and a sensor 260. The posture detection unit 204 detects the orientation of the terminal device 2. Specifically, the posture detection unit 204 acquires the information indicating the inclination and the information indicating the direction of the terminal device 2, and outputs the acquired information indicating the inclination and the information indicating the direction to the control unit 202. The control unit 202 acquires the information indicating the inclination and the information indicating the direction output by the attitude detection unit 204, and includes the acquired information indicating the inclination and the information indicating the direction in the garbage separation determination request. Further, the attitude detection unit 204 uses the information indicating the inclination and the information indicating the direction to perform image processing on the dust separation method explanatory information and the image information acquired by the image input unit 201 using the augmented reality technology. It is combined with the obtained image in real time. With this configuration, it is possible to improve the position accuracy when synthesizing the garbage separation method explanatory information into an image by using augmented reality technology. The position measurement unit 205 is realized by a command from the CPU 252 and a positioning unit 255. The position measuring unit 205 identifies the place where the terminal device 2 is used. The position measurement unit 205 measures the absolute position information of the terminal device 2 by the positioning unit 255. An example of absolute location information for a system is indicated by longitude and latitude. The position measurement unit 205 outputs the position information of the terminal device 2 to the control unit 202. The communication unit 206 is realized by a command from the CPU 252 and a communication I / F 166. The communication unit 206 acquires the garbage separation determination request output by the control unit 202, and transmits the acquired garbage separation determination request to the server 1 via the network 4. Further, the communication unit 206 receives the garbage separation response transmitted by the server 1 and outputs the received garbage separation response to the control unit 202. The image output unit 207 is realized by a command from the CPU 252 and a display unit 258. The image output unit 207 acquires the image information obtained by synthesizing the dust separation method explanatory information output by the control unit 202, and displays the image information combined with the acquired dust separation method explanatory information on the display unit 258.

(Operation of garbage separation support system)
An example of the operation of the garbage separation support system according to the embodiment of the present invention will be described with reference to FIG. In the example shown in FIG. 4, a method of separating dust is shown based on an image of dust obtained by imaging the dust and the trash can 5 with the terminal device 2 and imaging the dust, and an image of the trash can 5. Get information.
(Step S10)
The user of the terminal device 2 activates the garbage separation application by operating the operation unit 257.
(Step S11)
In the terminal device 2, when the dust separation application is activated, a message prompting the image of dust is displayed on the display unit 258.
(Step S12)
The user refers to the message displayed on the display unit 258 and performs an operation of capturing dust on the operation unit 257. As shown in FIG. 5, a PET bottle P is displayed on the display unit 258 as an example of dust to be imaged. The image input unit 201 causes the image pickup unit 259 to take an image of dust. The image input unit 201 outputs the image information of the dust obtained by imaging the dust to the control unit 202. The explanation will be continued by returning to FIG.
(Step S13)
The user performs an operation of imaging the trash can on the operation unit 257. As shown in FIG. 6, the display unit 258 shows a trash can 5 including three separate trash cans, a combustible trash can 51, a non-combustible trash can 52, and a resource trash can 53, as an example of the trash can 5 to be imaged. There is. The image input unit 201 causes the image pickup unit 259 to take an image of the trash can. The image input unit 201 outputs the image information of the trash can obtained by imaging the trash can to the control unit 202. The explanation will be continued by returning to FIG.
(Step S14)
The attitude detection unit 204 acquires information indicating the direction of the terminal device 2 based on the acceleration, geomagnetism, etc. detected by the sensor 260, and outputs the acquired position information of the terminal device 2 to the control unit 202.
(Step S15)
The position measurement unit 205 acquires the position information detected by the positioning unit 255, and outputs the acquired position information of the terminal device 2 to the control unit 202.
(Step S16)
The control unit 202 acquires the image information of the dust output by the image input unit 201 and the image information of the trash can, acquires the information indicating the direction of the terminal device 2 output by the attitude detection unit 204, and obtains the position measurement unit 205. Acquires the position information of the terminal device 2 output by. The control unit 202 requests a garbage separation determination including the acquired garbage image information, the trash can image information, the information indicating the orientation of the terminal device 2, the position information of the terminal device 2, and the identification information of the terminal device 2. Is created, and the created garbage separation determination request is output to the communication unit 206.
(Step S17)
The communication unit 206 acquires the garbage separation determination request output by the control unit 202, and transmits the acquired garbage separation determination request to the server 1.
(Step S18)
The server communication unit 101 receives the garbage separation determination request transmitted by the terminal device 2, and outputs the received garbage separation determination request to the terminal identification unit 102. The terminal identification unit 102 acquires the garbage separation determination request output by the server communication unit 101, and the image information of the garbage included in the acquired garbage separation determination request, the image information of the trash can, and the information indicating the orientation of the terminal device 2. And the position information of the terminal device 2 and the identification information of the terminal device 2 are acquired. The terminal identification unit 102 associates the acquired information indicating the orientation of the terminal device 2, the position information of the terminal device 2, and the identification information of the terminal device 2 with each other, and stores the information in the terminal information storage unit 103. Further, the terminal identification unit 102 image-recognizes the acquired image information of the dust, the image information of the trash can, the information indicating the orientation of the terminal device 2, the position information of the terminal device 2, and the identification information of the terminal device 2. Output to unit 104.
(Step S19)
The image recognition unit 104 includes image information of dust output by the terminal identification unit 102, image information of the garbage box 5, information indicating the orientation of the terminal device 2, position information of the terminal device 2, and identification information of the terminal device 2. And are acquired, and the feature amount of the dust image is extracted based on the acquired dust image information.
(Step S20)
The image recognition unit 104 extracts the feature amount of the image of the trash can based on the acquired image information of the trash can.

(Step S21)
The image recognition unit 104 acquires information indicating the sorting name of the dust corresponding to the feature amount of the extracted dust image by having the dust-specific feature amount storage unit 105 search.
(Step S22)
The image recognition unit 104 causes the trash can-specific feature amount storage unit 106 to search for the trash can configuration information corresponding to the feature amount of the extracted trash can image.
(Step S23)
The image recognition unit 104 estimates the location where the trash can 5 is installed based on the position information of the terminal device 2 and the information indicating the orientation of the terminal device 2. The image recognition unit 104 causes the area-specific separation information storage unit 107 to search for the identified garbage separation name and the area-specific separation stored in association with the estimation result of the place where the trash can 5 is installed. Get information. If there is regional sorting information corresponding to the identified sorting name and the estimation result of the place where the trash can 5 is installed, the regional sorting information storage unit 107 displays the corresponding regional sorting information as an image. Output to the recognition unit 104.
(Step S24)
The image recognition unit 104 causes the sorting method explanation information storage unit 108 to acquire the garbage separation method explanation information stored in association with the acquired sorting information for each region. If there is garbage separation method explanation information corresponding to the garbage separation method explanation information stored in association with the acquired area-specific separation information, the sorting method explanation information storage unit 108 corresponds to the garbage separation method explanation information. Is output to the image recognition unit 104.
(Step S25)
When the image recognition unit 104 acquires the garbage separation method explanation information output by the separation method explanation information storage unit 108, the image recognition unit 104 creates a garbage separation response including the acquired garbage separation method explanation information and the identification information of the terminal device 2. The created garbage separation response is output to the server communication unit 101 via the terminal identification unit 102.
(Step S26)
The server communication unit 101 transmits the created garbage separation response to the terminal device 2.
(Step S27)
The communication unit 206 receives the garbage separation response transmitted by the server 1 and outputs the received garbage separation response to the control unit 202.
(Step S28)
The control unit 202 acquires the garbage separation response output by the communication unit 206, and acquires the garbage separation method explanatory information included in the acquired garbage separation response. The control unit 202 processes the acquired dust separation method explanatory information using the augmented reality technology, and synthesizes the image information acquired by the image input unit 201 into the image obtained by the image processing in real time. As shown in FIG. 7, the PET bottle PI is displayed on the display unit 258 of the terminal device 2 in the image of the trash can 5 obtained by performing image processing on the image information of the trash can 5 acquired by the image input unit 201. .. Here, the label RaI is attached to the PET bottle PI. As shown in FIG. 8, the label RaI is removed while the PET bottle PI displayed on the display unit 258 of the terminal device 2 is displayed. As shown in FIG. 9, for each of the PET bottle PI displayed on the display unit 258 of the terminal device 2 and the label RaI, the three trash cans 5I displayed on the display unit 258 of the terminal device 2 Of the sorted trash cans, which of the sorted trash cans should be thrown away is indicated by an instruction image such as an arrow. Specifically, it is indicated that the PET bottle PI should be thrown into the resource trash can 53 and the label RaI should be thrown into the non-combustible trash can 52.

In the above-described embodiment, the case where the trash can 5 includes three separate trash cans of a combustible trash can 51, a non-combustible trash can 52, and a resource trash can 53 has been described, but the present invention is not limited to this example. For example, the trash can 5 may include at least one of a combustible trash can 51, a non-combustible trash can 52, and a resource trash can 53. Further, the trash can 5 is not limited to three trash cans 51, a non-combustible trash can 52, and a resource trash can 53, and may include four or more trash cans such as magazines, empty cans, and empty bottles. good.
In the above-described embodiment, the case where the dust and the trash can are imaged by the terminal device 2 has been described, but the present invention is not limited to this example. For example, the user may operate the terminal device 2 to take an image of the trash can 5 when the trash can 5 is full. The server 1 estimates the state of the trash can 5 based on the image information of the trash can transmitted by the terminal device 2, creates the state notification information including the information indicating the estimation result of the state of the trash can 5, and creates the state notification information. May be transmitted to an external device. With this configuration, predetermined processing such as collection of dust in the trash can 5 can be performed based on the status notification information notified to the external device. The efficiency of collecting the garbage in the trash can 5 can be improved, and the sanitary environment of the trash can 5 can be improved. Further, the configuration information of the trash can may be set in advance in the terminal device 2, and the imaging of the trash can may be omitted for the trash can having the same configuration as the set configuration information of the trash can. That is, the above-mentioned processing may be performed based on the image information of dust. With such a configuration, it is possible to eliminate the trouble of having the user take an image of the trash can, and it is possible to reduce the processing of the server 1.
In the above-described embodiment, the terminal device 2 may be provided with a storage unit that stores the image information of the garbage, the image information of the garbage box, and the information for explaining the garbage separation method in association with each other. Then, the control unit 202 acquires the garbage separation method explanatory information stored in association with the image information of the garbage input from the image input unit 201 and the image information of the garbage box from the storage unit, and the acquired garbage separation method. By processing the explanatory information using the augmented reality technique, the image information acquired by the image input unit 201 is combined with the image obtained by the image processing in real time. With this configuration, the process of inquiring the server 1 about the garbage separation method from the terminal device 2 can be omitted. Therefore, after the dust and the trash can are imaged, the garbage separation method is output from the terminal device 2. The time to complete can be shortened.
In the above-described embodiment, the image recognition unit 104 acquires and acquires the position information of the trash can in a predetermined range from the position of the terminal device 2 based on the acquired image information of the dust and the position information of the terminal device 2. You may create a garbage separation response that further includes the location information of the trash can. With this configuration, the user can be notified of the nearby trash can.
According to the garbage separation determination system of the present embodiment, by applying the extended reality technology and taking a picture of the garbage to be discarded and the garbage can with the image pickup unit, the garbage separation method on the display screen of the terminal device 2 is the garbage separation method. Displayed by method description information. Since the sorting method is displayed by sentences and animation, foreigners who are not accustomed to sorting garbage and tourists from different areas can fully understand the sorting method.

(Modification example)
In the above-described embodiment, the case where the garbage separation support system 3 includes the server 1 and the terminal device 2 has been described, but the present invention is not limited to this example. For example, the function of the server 1 may be configured to be included in the terminal device 2a. As shown in FIG. 10, the terminal device 2a includes an image input unit 201, a control unit 202a, an image recognition unit 203, an attitude detection unit 204, a position measurement unit 205, a communication unit 206, and an image output unit. It has an image recognition unit 204a, a garbage-specific feature amount storage unit 205a, a trash can-specific feature amount storage unit 206a, an area-specific sorting information storage unit 207a, and a sorting method explanation information storage unit 208a. FIG. 2 can be applied to the hardware configuration of the terminal device 2a. Each of these parts has a function realized by operating any of the components shown in FIG. 2 by an instruction from the CPU 252 that executes the program 2562 expanded from the non-volatile memory 256 to the memory 254. Is.
The control unit 202a is realized by an instruction from the CPU 252. The control unit 202a controls and manages the overall operation of the terminal device 2a. The control unit 202a acquires the image information output by the image input unit 201. Further, the control unit 202a acquires the position information of the terminal device 2a output by the position measurement unit 205. Further, the control unit 202a acquires the information indicating the direction of the terminal device 2a output by the attitude detection unit 204. The control unit 202a outputs the acquired image information, the position information of the terminal device 2a, and the information indicating the orientation of the terminal device 2a to the image recognition unit 204a.
Further, the control unit 202a has information on the garbage separation method output by the image recognition unit 204a, information indicating that there is no separation name, information indicating that there is no feature amount of the image of the garbage box, and information on sorting by region. Acquire either the information indicating that there is no garbage separation method or the information indicating that there is no garbage separation method explanation information. When the control unit 202a acquires the dust separation method explanatory information, the control unit 202a processes the dust separation explanatory information by using the augmented reality technology, and obtains the image information acquired by the image input unit 201 by image processing. It is combined with the image to be created in real time. The control unit 202a outputs the image information obtained by synthesizing the dust separation method explanatory information to the image output unit 207. When the control unit 202a acquires information other than the information for explaining the dust separation method, the control unit 202a processes the information by using the augmented reality technology, and obtains the image information acquired by the image input unit 201 by image processing. It is combined with the image to be created in real time. The control unit 202a outputs the image information obtained by synthesizing the information other than the information for explaining the dust separation method to the image output unit 207. Specifically, the control unit 202a causes the image recognition unit 203 to extract the feature amount from the image acquired by the image input unit 201. Further, the control unit 202a causes the posture detection unit 204 to detect the orientation of the terminal device 2a. The control unit 202a displays the garbage separation method explanation information so that the garbage separation method explanation information is displayed so as to overlap the image of the object such as the trash can 5 based on the feature amount extracted by the image recognition unit 203. Set the position to do. Further, the control unit 202a separates dust based on the orientation of the terminal device 2a detected by the posture detection unit 204 so that the dust separation method explanatory information is displayed so as to overlap the image of the object such as the trash can 5. Method Description Set the position to display the information.

The image recognition unit 204a is realized by a command from the CPU 252. The image recognition unit 204a acquires the position information of the terminal device 2a output by the terminal identification unit 102, the information indicating the posture of the terminal device 2a, the image information of the garbage, and the image information of the garbage box. The image recognition unit 204a analyzes the image of the garbage and the image of the garbage box by utilizing image processing technology or the like based on the acquired image information of the garbage and the image information of the garbage box. , Extract the feature quantity. The image recognition unit 204a acquires information indicating the separated name stored in association with the feature amount of the extracted dust image by having the dust-specific feature amount storage unit 205a search. The image recognition unit 204a causes the garbage-specific feature amount storage unit 205a to acquire detailed sorting names such as PET bottles and cans. The image recognition unit 204a acquires information indicating a sorting name corresponding to the dust of the subject of the image output by the dust-specific feature amount storage unit 205a. Further, the image recognition unit 204a acquires information from the dust-specific feature amount storage unit 105 indicating that there is no feature amount of the dust image that matches the feature amount of the dust image of the subject of the image. The image recognition unit 204a causes the trash can-specific feature amount storage unit 206a to acquire the configuration information of the trash can that matches the feature amount of the image of the trash can extracted by searching. The image recognition unit 204a acquires the configuration information of the trash can output by the feature amount storage unit 206a for each trash can. The image recognition unit 204a acquires information from the trash can-specific feature amount storage unit 206a indicating that there is no feature amount of the trash can image that matches the feature amount of the trash can image of the subject of the image.
The image recognition unit 204a estimates the location where the trash can 5 is installed based on the position information of the terminal device 2a and the information indicating the orientation of the terminal device 2a. The image recognition unit 204a causes the area-specific separation information storage unit 207a to search for the identified garbage separation name and the area-specific separation stored in association with the estimation result of the place where the trash can 5 is installed. Get information. The image recognition unit 204a acquires the regional classification information output by the regional classification information storage unit 207a. The image recognition unit 204a acquires information indicating that there is no regional classification information from the regional classification information storage unit 207a. The image recognition unit 204a causes the sorting method explanation information storage unit 208a to search for the garbage separation method explanation information stored in association with the acquired sorting information for each region. The image recognition unit 204a acquires the garbage separation method explanation information output by the separation method explanation information storage unit 208a. The image recognition unit 204a acquires information indicating that there is no garbage separation method explanation information from the separation method explanation information storage unit 208a.

When the image recognition unit 204a acquires the garbage separation method explanation information output by the separation method explanation information storage unit 208a, the image recognition unit 204a outputs the acquired garbage separation method explanation information to the control unit 202a. Further, when the image recognition unit 204a acquires information indicating that there is no feature amount of the dust image that matches the feature amount of the dust image of the subject of the image output by the dust-specific feature amount storage unit 205a, the image recognition unit 204a obtains information. Information indicating that there is no sorting name is output to the control unit 202a. Further, when the image recognition unit 204a acquires information indicating that there is no feature amount of the image of the trash box that matches the feature amount of the image of the trash box of the subject of the image output by the feature amount storage unit 206a for each trash box, the image recognition unit 204a obtains information. Information indicating that there is no feature amount of the image of the trash box that matches the feature amount of the image of the trash box of the subject of the image is output to the control unit 202a. Further, when the image recognition unit 204a acquires the information indicating that there is no regional classification information output by the regional classification information storage unit 207a, the image recognition unit 204a outputs the information indicating that there is no regional classification information to the control unit 202a. do.
The garbage-specific feature amount storage unit 205a, the garbage box-specific feature amount storage unit 206a, the area-specific sorting information storage unit 207a, and the sorting method explanation information storage unit 208a are the garbage-specific feature amount storage unit 105 described in the above-described embodiment, respectively. The feature amount storage unit 106 for each garbage box, the sorting information storage unit 107 for each area, and the information storage unit 108 for explaining the sorting method can be applied.
According to the terminal device 2a of the modified example, as compared with the terminal device 2 of the embodiment, the user can be notified of the garbage separation method without communicating with the server 1.
Although the embodiments have been described above, these embodiments are presented as examples and are not intended to limit the scope of the invention. These embodiments can be implemented in various other forms, and various omissions, replacements, changes, and combinations can be made without departing from the gist of the invention. These embodiments are included in the scope and gist of the invention, and at the same time, are included in the scope of the invention described in the claims and the equivalent scope thereof. The above-mentioned garbage separation support system 3 may be realized by a computer. In that case, a program for realizing the function of each functional block is recorded on a computer-readable recording medium. The program recorded on the recording medium may be read by the computer system and executed by the CPU. The term "computer system" as used herein includes hardware such as an OS (Operating System) and peripheral devices.

Further, the "computer-readable recording medium" refers to a portable medium such as a flexible disk, a magneto-optical disk, a ROM, or a CD-ROM. Further, the "computer-readable recording medium" includes a storage device such as a hard disk built in a computer system. Further, the "computer-readable recording medium" may include a medium that dynamically holds the program for a short period of time. What dynamically holds the program for a short period of time is, for example, a communication line when the program is transmitted via a network such as the Internet or a communication line such as a telephone line.
Further, the "computer-readable recording medium" may include a medium that holds a program for a certain period of time, such as a volatile memory inside a computer system serving as a server or a client. Further, the above program may be for realizing a part of the above-mentioned functions. Further, the above-mentioned program may be realized by combining the above-mentioned functions with a program already recorded in the computer system. Further, the above program may be realized by using a programmable logic device. The programmable logic device is, for example, an FPGA (Field Programmable Gate Array). The above-mentioned garbage separation support system 3 has a computer inside. The processing processes of the server 1, the terminal device 2, and the terminal device 2a described above are stored in a computer-readable recording medium in the form of a program, and the computer reads and executes the program to perform the above-mentioned processing. Processing is done. Here, the computer-readable recording medium refers to a magnetic disk, a magneto-optical disk, a CD-ROM, a DVD-ROM, a semiconductor memory, or the like. Further, this computer program may be distributed to a computer via a communication line, and the computer receiving the distribution may execute the program.
Further, the above program may be for realizing a part of the above-mentioned functions.
Further, it may be a so-called difference file (difference program) that can realize the above-mentioned function in combination with a program already recorded in the computer system.

1 ... Server, 2 ... Terminal device, 3 ... Garbage separation support system, 4 ... Network, 5 ... Garbage box, 51 ... Combustible garbage box, 52 ... Non-combustible garbage box, 53 ... Resource garbage box, 101 ... Server communication unit, 102 ... Terminal identification unit , 103 ... Terminal information storage unit, 104, 204a ... Image recognition unit, 105, 205a ... Garbage-specific feature amount storage unit, 106, 206a ... Garbage box-specific feature amount storage unit, 107, 207a ... Regional sorting information storage unit, 108 , 208a ... Separation method explanation information storage unit, 201 ... image input unit, 202, 202a ... control unit, 104, 203, 204a ... image recognition unit, 204 ... attitude detection unit, 205 ... position measurement unit, 206 ... communication unit, 207 ... Image output unit, 152, 252 ... CPU, 154, 254 ... Memory, 156, 256 ... Non-volatile memory, 162, 257 ... Operation unit, 166, 266 ... Communication I / F, 255 ... Positioning unit, 260 ... Sensor 258 ... Display unit, 259 ... Imaging unit

Claims (6)

It is a garbage separation support system equipped with a terminal device and a server.
The terminal device is
An image input unit that acquires image information of garbage,
Positioning unit that positions the position of the own terminal device and
A dust separation determination request, which is a signal including the image information of the dust acquired by the image input unit and the position information of the own terminal device positioned by the positioning unit, is requested to determine the separation of the dust. The control unit to be created and
A communication unit that transmits the garbage separation determination request created by the control unit to the server and receives a garbage separation response that is a response signal to the transmitted garbage separation determination request.
The image information indicating the dust separation method included in the dust separation response received by the communication unit is acquired, and based on the acquired image information indicating the dust separation method, an image showing the dust separation method and the said It is provided with an image output unit that synthesizes and displays the image of the trash can obtained from the image information of the trash can acquired by the image input unit.
The server
The image information of the dust included in the dust separation determination request transmitted by the terminal device and the position information of the own terminal device are acquired, and based on the acquired image information of the dust, the dust of the dust The feature amount of the image is extracted, and the garbage separation name corresponding to the feature amount of the extracted garbage image is associated with the feature amount of the garbage image and the garbage separation name shown in the garbage image. Based on the separated name of the garbage acquired from the stored database and the position information of the own terminal device, information indicating the method of separating the garbage is acquired, and the acquired method of separating the garbage is obtained. An image recognition unit that acquires image information indicating the method for separating the dust based on the indicated information and creates the dust separation response including the acquired image information indicating the method for separating the dust.
A server communication unit that transmits the garbage separation response created by the image recognition unit to the terminal device is provided .
The image input unit acquires the image information of the trash can and obtains the image information.
The control unit creates the garbage separation determination request further including the image information of the trash can.
The image recognition unit further acquires the image information of the trash can included in the trash separation determination request transmitted by the terminal device, and obtains the image information of the trash can and the position information of the own terminal device. Based on, the position information of the trash can is acquired, and
Obtained on the basis of the said fractionation name and the position information of the trash garbage, you acquire information indicating the method of separating the waste, garbage separation support system. The terminal device is
It is equipped with a posture detection unit that detects the orientation of the own terminal device.
The image output unit is the dust to be combined with the image of the trash can obtained from the image information of the trash can acquired by the image input unit based on the information indicating the orientation detected by the posture detection unit. The garbage sorting support system according to claim 1, which corrects the position of an image showing a sorting method. The image recognition unit acquires the position information of the trash can within a predetermined range from the position of the own terminal device based on the acquired image information of the dust and the position information of the own terminal device.
The garbage separation support system according to claim 1 or 2 , wherein the server communication unit creates the garbage separation response further including the position information of the trash can acquired by the image recognition unit. An image input unit that acquires image information of garbage and image information of a trash can,
Positioning unit that positions the position of the own terminal device and
Based on the image information of the dust acquired by the image input unit, the feature amount of the dust image is extracted, and the sorting name of the dust corresponding to the feature amount of the extracted dust image is given to the dust. Obtained from a database that stores the feature amount of the image and the garbage sorting name indicated by the garbage image in association with each other, the obtained garbage sorting name , the garbage box image information, and the self positioned by the positioning unit. Based on the position information of the garbage box acquired based on the position information of the terminal device, information indicating the garbage sorting method is acquired, and based on the acquired information indicating the garbage sorting method, the said An image recognition unit that acquires image information indicating the method for separating dust, and
Based on the image information indicating the sorting method of the dust acquired by the image recognition unit, the image of the trash can obtained from the image showing the sorting method of the dust and the image information of the trash can acquired by the image input unit. A terminal device including an image output unit that synthesizes and displays the image. It is a garbage separation support method executed by a garbage separation support system equipped with a terminal device and a server.
The step in which the terminal device acquires the image information of the garbage and the image information of the garbage box,
A step in which the terminal device determines the position of the own terminal device,
The terminal device creates a garbage separation determination request which is a signal including the image information of the dust, the image information of the trash can, and the position information of the own terminal device, and which is a signal requesting the determination of the garbage separation. Steps to do and
A step in which the terminal device transmits the garbage separation determination request to the server,
A step in which the server acquires the image information of the garbage included in the garbage separation determination request transmitted by the terminal device, the image information of the trash can, and the position information of the own terminal device.
The server extracts the feature amount of the dust image based on the image information of the dust, and names the garbage separation corresponding to the feature amount of the extracted dust image as the feature of the dust image. Steps to obtain from the database that stores the amount and the garbage classification name shown in the garbage image in association with each other,
The server shows the garbage separation method based on the garbage separation name , the position information of the trash can acquired based on the image information of the trash can and the position information of the own terminal device. Steps to get information and
A step in which the server acquires image information indicating the method for separating the garbage based on the information indicating the method for separating the garbage.
A step in which the server creates a garbage separation response including the image information indicating the method for separating the garbage.
A step in which the server transmits the garbage separation response to the terminal device,
A step in which the terminal device receives the garbage separation response transmitted by the server, and
The terminal device acquires the image information indicating the method for separating the dust included in the dust separation response, and based on the image information indicating the acquired method for separating the dust, the method for separating the dust. A garbage separation support method comprising a step of synthesizing and displaying an image showing the above and the image of the trash can obtained from the acquired image information of the trash can. To the computer of the server that communicates with the terminal device
A step of acquiring the image information of the garbage included in the garbage separation determination request transmitted by the terminal device, the image information of the trash can, and the position information of the terminal device.
Based on the acquired image information of the dust, the feature amount of the dust image is extracted, and the sorting name of the dust corresponding to the feature amount of the extracted dust image is referred to as the feature amount of the dust image. The steps to obtain from the database stored in association with the garbage classification name shown in the garbage image, and
Based on the acquired sorting name of the trash, the image information of the trash can, and the position information of the trash can acquired based on the position information of the terminal device, information indicating the trash sorting method is provided. Steps to get and
Based on the acquired information indicating the method for separating the garbage, a step of acquiring image information indicating the method for separating the garbage, and
A program for creating a dust separation response including the image information indicating the method for separating the dust, and executing a step of transmitting the created dust separation response to the terminal device.

Images