JP2021114335A - Database generation method, additional information registration system, database generation device and program - Google Patents

Abstract

To provide a database generation method, a database generation device and a database generation program which can enhance the segmentation degree of classification labels while efficiently increasing the sample number of teacher data.SOLUTION: A database generation method comprises: a position detection step of detecting an imaging position at the time when a prescribed subject is imaged in order to acquire an image of the subject; a first determination step of determining whether or not the detected imaging position is closer than a prescribed distance threshold from a prescribed registered point; a second determination step of determining whether or not the subject is a previously-registered object when determining that the imaging position is closer than the prescribed distance threshold from the prescribed registered point; a display step of displaying a screen to which a user can input additional information to be registered in association with the image when determining that the subject is the object; and a registration step of registering the additional information input by the user in prescribed storage means as a classification label.SELECTED DRAWING: Figure 4

Description

The present invention relates to a database generation method, a database generation device, and a database generation program.

In recent years, in supervised learning, which is a method of machine learning, a method of classifying (discriminating) the contents of an image using a classifier (discriminant) generated as a result of learning has been known (for example, a patent). Reference 1).

Japanese Unexamined Patent Publication No. 2008-282267

However, it has been difficult to efficiently collect data when increasing the number of samples of teacher data and subdividing the classification labels registered in association with the teacher data.

Therefore, an object of the present invention is to provide a database generation method, a database generation device, and a database generation program capable of increasing the degree of subdivision of classification labels while efficiently increasing the number of samples of teacher data.

In order to achieve the above object, the database generation method according to the present invention is a database generation method executed by the additional information registration system, and when the subject is imaged in order to acquire an image of a predetermined subject. A position detection step for detecting an imaging position, a first determination step for determining whether or not the imaging position detected by the position detection step is closer than a predetermined distance threshold value from a predetermined registration point, and the first determination step. When it is determined by the determination step that the imaging position is closer than the predetermined distance threshold value from the predetermined registration point, it is determined whether or not the subject is an object registered in advance corresponding to the predetermined registration point. When the subject is determined to be the object by the second determination step and the second determination step, additional information for registering the subject in association with the image is used as answer information for a predetermined question. , When the answer information is input to the display step for displaying the question screen that can be input by the user and the question screen displayed by the display step, the input answer information is designated as a classification label. It is characterized by having a registration step of registering in a storage means.

Further, the database generation method of another aspect according to the present invention is a database generation method executed by the additional information registration system, which is based on an acquisition step of acquiring an image of the subject by imaging a predetermined subject and the acquisition step. A position detection step for detecting an imaging position when the subject is imaged, and a first determination for determining whether or not the imaging position detected by the position detection step is closer than a predetermined distance threshold value from a predetermined registration point. When it is determined by the determination step and the first determination step that the imaging position is closer than the predetermined distance threshold value from the predetermined registration point, the subject is registered in advance corresponding to the predetermined registration point. When the subject is determined to be the object by the second determination step for determining whether or not the subject is an object and the second determination step, the subject is associated with the image acquired by the acquisition step. The answer information is input to the display step for displaying the question screen that can be input by the user as the answer information for the predetermined question as the additional information for registration, and the question screen displayed by the display step. If so, it is characterized by having a registration step of registering the input response information as a classification label in a predetermined storage means.

Further, the database generation device according to the present invention includes a position detecting means for detecting an imaging position when the subject is imaged in order to acquire an image of a predetermined subject, and the imaging position detected by the position detecting means. The first determination means for determining whether or not is closer to the predetermined distance threshold from the predetermined registration point, and the first determination means for determining that the imaging position is closer than the predetermined distance threshold from the predetermined registration point. If so, the subject is the object by the second determination means for determining whether or not the subject is an object registered in advance corresponding to the predetermined registration point, and the second determination means. When it is determined that the information is, the display control means for displaying the question screen on which the user can input additional information for registering the image in association with the image as the answer information for the predetermined question, and the display. When the answer information is input to the question screen displayed by the control means, the registration means for registering the input answer information as a classification label in a predetermined storage means is provided.

In addition, the database generation device of another aspect according to the present invention has an acquisition means for acquiring an image of the subject by imaging a predetermined subject, and a position for detecting an imaging position when the subject is imaged by the acquisition means. The detection means, a first determination means for determining whether or not the imaging position detected by the position detection means is closer than a predetermined distance threshold from a predetermined registration point, and the imaging position by the first determination means. Is a second determination means for determining whether or not the subject is a pre-registered object corresponding to the predetermined registration point when it is determined that the subject is closer than the predetermined distance threshold from the predetermined registration point. When the subject is determined to be the object by the second determination means, additional information for registering the image acquired by the acquisition means in association with the image is provided as answer information to a predetermined question. When the answer information is input to the display control means for displaying the question screen that can be input by the user and the question screen displayed by the display control means, the input answer information is classified as a classification label. It is characterized in that it includes a registration means for registering in a predetermined storage means.

In addition, the database generation program according to the present invention is a position detecting means for detecting an imaging position when the subject is imaged in order to acquire an image of a predetermined subject, and the position detecting means detected by the position detecting means. When the imaging position is closer than the predetermined distance threshold from the predetermined registration point by the first determination means for determining whether or not the imaging position is closer than the predetermined distance threshold from the predetermined registration point, the first determination means. When the determination is made, the subject is the object by the second determination means for determining whether or not the subject is an object registered in advance corresponding to the predetermined registration point, and the second determination means. Display control means for displaying a question screen that can be input by the user as answer information for a predetermined question with additional information for registering in association with the image when the determination is made. When the answer information is input to the question screen displayed by the means, the input answer information is made to function as a registration means for registering the input answer information as a classification label in a predetermined storage means.

In addition, the database generation program of another aspect according to the present invention detects an acquisition means for acquiring an image of the subject by imaging a predetermined subject, and an imaging position when the subject is imaged by the acquisition means. Position detecting means, a first determining means for determining whether or not the imaging position detected by the position detecting means is closer than a predetermined distance threshold from a predetermined registration point, and the imaging position by the first determining means. Is a second determination means for determining whether or not the subject is a pre-registered object corresponding to the predetermined registration point when it is determined that the subject is closer than the predetermined distance threshold from the predetermined registration point. When the subject is determined to be the object by the second determination means, additional information for registering the subject in association with the image acquired by the acquisition means is used as answer information for a predetermined question. , A display control means for displaying a question screen that can be input by the user, and when the answer information is input to the question screen displayed by the display control means, the input answer information is designated as a classification label. It is characterized in that it functions as a registration means for registering in the storage means of the above.

According to the present invention, it is possible to increase the degree of subdivision of the classification label while efficiently increasing the number of samples of teacher data.

The figure which shows the outline of the additional information registration system which concerns on Embodiment 1. The figure which shows the physical configuration of the server which concerns on Embodiment 1. FIG. The figure which shows the physical structure of the mobile terminal which concerns on Embodiment 1. FIG. The figure which shows the functional structure of the additional information registration system which concerns on Embodiment 1. FIG. The figure which shows the example of the data structure of the line database which concerns on Embodiment 1. The figure which shows the example of the data structure of the object database which concerns on Embodiment 1. The figure which shows the example of the data structure of the location information database which concerns on Embodiment 1. The figure which shows the example of the data structure of the re-learning database which concerns on Embodiment 1. The figure which shows the map of the park where the object which concerns on Embodiment 1 is arranged. The figure which shows the map of the park which displayed the line stored in the line database which concerns on Embodiment 1. The figure which shows the map of the park which displayed the line stored in the line database, the object stored in the object database, and the registration point associated with the object which concerns on Embodiment 1. FIG. The figure which shows the start-up screen displayed on the display part of the mobile terminal which concerns on Embodiment 1. FIG. The figure which shows the camera shooting screen displayed on the display part of the mobile terminal which concerns on Embodiment 1. FIG. FIG. 5 is a diagram showing a camera shooting screen after capturing an image of a subject displayed on the display unit of the mobile terminal according to the first embodiment. The figure which shows the text / photograph display screen displayed on the display part of the mobile terminal which concerns on Embodiment 1. FIG. FIG. 5 is a diagram showing a text / photo display screen during natural language dialogue processing displayed on the display unit of the mobile terminal according to the first embodiment. The figure which shows the text / photograph display screen after the natural language dialogue processing displayed on the display part of the mobile terminal which concerns on Embodiment 1. FIG. FIG. 5 is a diagram showing a text / photo display screen during additional information input processing in which the first question is displayed on the display unit of the mobile terminal according to the first embodiment. FIG. 5 is a diagram showing a text / photo display screen during additional information input processing in which the last question is displayed on the display unit of the mobile terminal according to the first embodiment. The figure which shows the map display screen displayed on the display part of the mobile terminal which concerns on Embodiment 1. FIG. The flowchart which shows the additional information registration process which concerns on Embodiment 1. The flowchart which shows the route collation processing which concerns on Embodiment 1. The figure for demonstrating the foot of the perpendicular line on the line with respect to the imaging position which concerns on Embodiment 1. FIG. The flowchart which shows the guide object search process which concerns on Embodiment 1. The flowchart which shows the additional information input processing which concerns on Embodiment 1. The flowchart which shows the classification label registration process which concerns on Embodiment 1. The figure which shows the physical structure of the mobile terminal which concerns on Embodiment 2. The flowchart which shows the additional information registration process which concerns on Embodiment 2.

(Embodiment 1)
Hereinafter, embodiments for carrying out the present invention (Embodiment 1) will be described in detail with reference to the drawings.

FIG. 1 is a diagram showing an outline of the additional information registration system 10 according to the first embodiment. The additional information registration system 10 includes a server 100 and a mobile terminal 200 capable of communicating with each other on the network 500.

When the user who owns the mobile terminal 200 images a plant as a subject in a predetermined place such as a predetermined park, the additional information registration system 10 displays guidance information related to the plant on the display unit of the mobile terminal 200. It is assumed that the system can provide information to the user by displaying the information on the user.

FIG. 2 is a diagram showing a physical configuration of the server 100. As shown in FIG. 2, the server 100 includes a CPU (Central Processing Unit) 110, a RAM (Random Access Memory) 121, a storage unit 130, an operation unit 122, a display unit 123, and a communication unit 124. ing. Further, each part of the server 100 is connected via the bus 129.

The server 100 is, for example, a server device of a server / client system using a network, but the present invention is not limited to this, and a virtual machine operating on a cloud system may be used.

The CPU 110 controls each part of the server 100. The CPU 110 reads a designated program among the system programs and application programs stored in the storage unit 130, expands the program into the work area of the RAM 121, and executes various processes according to the program. For example, the CPU 110 functions as a first determination means, a second determination means, and a registration means by executing an additional information registration process described later. Further, the CPU 110 has a function of executing additional information registration processing described later in cooperation with the mobile terminal 200.

The RAM 121 is, for example, a volatile memory and has a work area for temporarily storing various programs and data read by the CPU 110.

The storage unit 130 is composed of, for example, an HDD (Hard Disk Drive), an SSD (Solid State Drive), an EEPROM (Electrical Erasable Programmable Read-Only Memory), a flash memory, or the like, and can write data and a program. Is. The storage unit 130 contains a system program and various processing programs executed by the CPU 110, data necessary for executing these programs, a line database (Date Base) 140 described later, an object database 150 described later, and a location information database described later. 170, the re-learning database 180, which will be described later, and the like are stored.

The operation unit 122 is composed of, for example, a key input unit such as a keyboard and a pointing device such as a mouse. Further, the operation unit 122 receives the key input and the position input, and outputs the operation information as an input signal to the CPU 110.

The display unit 123 is composed of, for example, an LCD (Liquid Crystal Display), an organic EL (Electroluminescence) display, or the like. Further, the display unit 123 displays various screens according to the instruction of the display signal input from the CPU 110.

The communication unit 124 is composed of a network card or the like and is connected to an external device on the network 500. The network 500 includes a LAN (Local Area Network), a WAN (Wide Area Network), and the like. The server 100 can communicate with the mobile terminal 200 on the network 500 via the communication unit 124.

FIG. 3 is a diagram showing a physical configuration of the mobile terminal 200. As shown in FIG. 3, the mobile terminal 200 includes a CPU 210, a RAM 221, a storage unit 230, an operation unit 222, a display unit 223, a position information calculation unit 224, an imaging unit 225, and a voice input unit 226. , A voice output unit 227 and a communication unit 228 are provided. Further, each part of the mobile terminal 2 is connected via a bus 229.

The mobile terminal 200 is a mobile terminal device capable of wireless communication. Here, it is assumed that the mobile terminal 200 is a smartphone, but the present invention is not limited to this, and the mobile terminal 200 may be another portable terminal device.

The CPU 210 controls each part of the mobile terminal 200. The CPU 210 reads a designated program among the system programs and application programs stored in the storage unit 230, expands the program in the work area of the RAM 221 and executes various processes according to the program. At that time, the CPU 210 stores various processing results in the RAM 221 and displays the processing results on the display unit 223 as needed. For example, the CPU 210 functions as a position detecting means and a display controlling means by executing an additional information registration process described later.

The RAM 221 is, for example, a volatile memory and has a work area for temporarily storing various programs and data read by the CPU 210.

The storage unit 230 is a storage unit that is composed of, for example, an HDD, an SSD, an EEPROM, a flash memory, or the like, and is capable of writing and reading data and programs. Further, the storage unit 230 stores application programs such as the additional information registration program 231 executed by the CPU 210, the various programs described above, data necessary for executing these programs, and the like.

The operation unit 222 includes various function keys, receives press input of each key by the user, and outputs the operation information to the CPU 210. Further, the operation unit 222 has a touch panel or the like in which transparent electrodes are arranged in a grid pattern so as to cover the surface of the display unit 223, detects a position pressed by a finger, a touch pen, or the like, and uses the position information as operation information. Output to CPU 210.

The display unit 223 is composed of an LCD, an organic EL display, or the like. The display unit 223 displays various screens according to the instructions of the display signal input from the CPU 210.

The position information calculation unit 224 is composed of an antenna, a reception circuit, a signal processing circuit, and the like, and receives a radio wave transmitted from a positioning satellite related to the global positioning system to acquire the current position of the mobile terminal 200. The position information calculation unit 224 amplifies, frequency-converts, and decodes the received transmitted radio wave to acquire necessary information and outputs it to the CPU 210.

The image pickup unit 225 is composed of an optical system, a camera including an image pickup element, and the like. The image sensor is, for example, an image sensor such as a CCD (Charge Coupled Device) or a CMOS (Complementary Metal Oxide Semiconductor). An optical image of the subject is formed on the image sensor, and the image is output to the CPU 210 as an electrical signal.

The voice input unit 226 is composed of a microphone, an A / D converter, and the like, and outputs the sound input from the microphone to the CPU 210 as a digital signal.

The audio output unit 227 is composed of a speaker (earphone), a D / A converter, and the like, and outputs a digital signal input from the CPU 210 as sound.

The communication unit 228 is composed of, for example, a network card, a modem, a router, and the like. Further, the communication unit 228 may be provided with an antenna, a modulation / demodulation circuit, a signal processing circuit, and the like, and may be configured to enable wireless communication. The communication unit 228 communicates with an external device such as a server 100 on the network 500 to transmit / receive data.

FIG. 4 is a diagram showing a functional configuration of the additional information registration system 10. As shown in FIG. 4, the additional information registration system 10 includes an acquisition unit 311, a position detection unit 312, a first determination unit 313, a second determination unit 314, a display control unit 315, and a registration unit 316. And. These function as acquisition means, position detection means, first determination means, second determination means, display control means, and registration means, respectively.

The acquisition unit 311 acquires, for example, the subject when the user who owns the mobile terminal 200 captures the subject as an image.

The position detection unit 312 detects, for example, the imaging position when the user who owns the mobile terminal 200 images the subject.

The first determination unit 313 determines whether or not the imaging position detected by the position detection unit 312 is within a predetermined range from the registration point described later.

The second determination unit 314 determines, for example, whether or not the subject imaged by the user who owns the mobile terminal 200 is an object registered in advance in the object database described later.

When the second determination unit 314 determines that the subject is an object, the display control unit 315 displays, for example, a screen on which additional information for registering the subject in association with the subject can be input. Display on.

The registration unit 316 registers the additional information input to the screen displayed by the display control unit 315 as a classification label indicating what the image of the subject represents or what is included in the image.

FIG. 5 shows an example of the data structure of the line database 140. The line database 140 is, for example, tabular data, one row (record) indicates one line, data item 141 of line identification information (described as line ID (Identifier) in FIG. 5), start point longitude. Includes columns of data item 142, start point latitude data item 143, end point longitude data item 144, and end point latitude data item 145. The line is a preset line (route, route) that the user who owns the mobile terminal 200 is expected to pass when viewing the plant in the park. For example, the line is a flower bed in which the plant is planted. It is set along the perimeter. Further, the start point position and the end point position of the line are determined based on the direction assumed to pass when the user moves.

The data item 141 of the line identification information includes the line identification information indicated by the record.

The data item 142 of the start point longitude and the data item 143 of the start point latitude include the longitude and latitude of the start point of the line indicated by the record.

The end point longitude data item 144 and the end point latitude data item 145 include the end point longitude and latitude of the line indicated by the record.

For example, record 149 indicates a line in which the data item 141 of the line identification information is "L1", the starting point is 139.5465128 degrees east longitude and 35.66983 degrees north latitude, and the end point is 139.54633939 degrees east longitude and 35 north latitude. It shows that it is .6691442 degrees.

FIG. 6 shows an example of the data structure of the object database 150. The object database 150 is, for example, tabular data, one row (record) indicates one object, data item 151 of object identification information (described as object ID in FIG. 6), longitude. Data item 152, latitude data item 153, registration point longitude data item 154, registration point latitude data item 155, line data item 156 where registration point is set, block data item 157, name data item 158. , Image data item 159, color data item 160 (denoted as COLOR in FIG. 6) and shape data item 161 (denoted as SHAPE in FIG. 6). In the first embodiment, the object refers to a plant planted in the park and is registered in the object database 150 in advance. As will be described later, the object database 150 identifies the names of plants and the like included in the photographed image when the user images the subject through the image pickup unit 225 of the mobile terminal 200 in the park, and guide information. Is referred to for display on the display unit 223. Further, as will be described later, the block indicates a block (flowerbed) in which a plant such as a rose, which is an object, is planted. The object database 150 may include other data items such as the type of object (TYPE).

The data item 151 of the object identification information includes the object identification information indicated by the record.

The longitude data item 152 and the latitude data item 153 include the longitude and latitude of the position of the object indicated by the record.

The data item 154 of the registration point longitude and the data item 155 of the registration point latitude include the longitude and latitude of the imaging spot suitable for imaging the object indicated by the record. Hereinafter, the position specified from these two data items will be referred to as a registration point. In addition, the registration point is set on the above-mentioned line, that is, it is set at a position that is expected to pass when viewing plants in the park.

The data item 156 of the line includes a line in which a registration point, which is an imaging spot of the object indicated by the record, is set.

The data item 157 of the block includes the block number in which the object indicated by the record is arranged.

The name data item 158 includes the name of the object indicated by the record.

The image data item 159 includes identification information of the image data of the object indicated by the record. The image data is stored in, for example, the storage unit 130 of the server 100.

The color data item 160 includes the color of the object indicated by the record.

The shape data item 161 includes the shape of the object indicated by the record.

For example, the record 169 is a record indicating an object whose data item 151 of the object identification information is "T1", and the position of the object is 139.5465239 degrees east longitude, 35.6697512 degrees north latitude, and "B5". The name of the object is "Sonyahorstmann", the identification information of the image is "G7872", and it is red, indicating that it has multiple shapes. In addition, the registration points are 139.5465347 degrees east longitude and 35.66697598 degrees north latitude, indicating that they are set on line L10.

FIG. 7 shows an example of the data structure of the location information database 170. The location information database 170 is, for example, tabular data, and one row (record) indicates information related to one location information message transmitted by the mobile terminal 200 and received and stored by the server 100. It is composed of columns of identification information data item 171 (denoted as position ID in FIG. 7), terminal data item 172, current longitude data item 173, current latitude data item 174, and movement direction data item 175. NS.

The data item 171 of the position identification information includes the identification information of the position information indicated by the record.

The data item 172 of the terminal includes the identification information of the mobile terminal 200 that has transmitted the location information message that is the source of the record.

The data item 173 of the current longitude and the data item 174 of the current latitude include the longitude and latitude included in the position information message.

The movement specified by the data item 175 in the moving direction includes the current position information indicated by the data item 173 of the current longitude and the data item 174 of the current latitude, and the past position information (not shown) of the mobile terminal 200. The direction is included. In the first embodiment, the data item 175 in the moving direction includes "E" indicating the movement in the east direction, "W" indicating the movement in the west direction, "S" indicating the movement in the south direction, and "S" indicating the movement in the north direction. It includes, but is not limited to, the four directions of "N" indicating the movement of. Eight directions including "NE" indicating northeast movement, "NW" indicating northwest movement, "SE" indicating southeast movement, and "SW" indicating southwest movement. May be
"NNE" indicating movement in the north-northeast direction, "NNW" indicating movement in the north-northwest direction, "ESE" indicating movement in the east-southeast direction, movement in the west-southwest direction indicating movement in the east-northeast direction The sixteen directions may further include "WSW" indicating movement in the west-northwest direction, "WNW" indicating movement in the west-northwest direction, "SSE" indicating movement in the south-southeast direction, and "SSW" indicating movement in the south-southwest direction.

In the position identification information data item 171, the terminal data item 172, the current longitude data item 173, the current latitude data item 174, and the moving direction data item 175 described above, the server 100 sends a position information message. When it is received, it is added to the location information database 170.

For example, the record 179 is a record in which the data item 171 of the position identification information is "P1", and the position information included in the position information message transmitted from the mobile terminal 200 whose identification information is "4673" is included in the position information message. , 139.5465239 degrees east longitude and 35.6697512 degrees north latitude, indicating that the user has moved toward the south.

FIG. 8 shows an example of the data structure of the retraining database 180. The re-learning database 180 includes additional information input by the user in the additional information input process described later. The re-learning database 180 is, for example, tabular data, one row (record) indicates one guide object, and data item 181 of the guide object identification information (described as the guide object ID in FIG. 8). ), Object identification information data item 182, image data item 183, color data item 184 (described as COLOR in FIG. 8), shape data item 185 (described as SHAPE in FIG. 8), flower state data. It is composed of columns (attributes) such as item 186, data item 187 for the number of petals, and data item 188 for leaf shape.

Here, the guide object indicates a plant to which the guide information determined based on the current position of the mobile terminal 200, the captured image, or the like should be transmitted. In the first embodiment, the guidance information relating to the plant determined as the guidance target is transmitted, and the user of the mobile terminal 200 is made to register additional information about each plant in association with the target database 150 of FIG. It has become. Thereby, the information contained in the object database 150, that is, the information referred to for determining the plant based on the captured image can be easily accumulated.

The data item 181 of the guide object identification information includes the guide object identification information indicated by the record.

The data item 182 of the object identification information includes an object ID for identifying which of the objects in the object database 150 the guide object indicated by the record is.

The image data item 183 includes identification information of image data acquired by the user taking an image of the guide object indicated by the record.

The color data item 184 includes the color of the guide object indicated by the record as additional information input by the user in the additional information input process described later.

The shape data item 185 includes the shape of the guide object indicated by the record as additional information input by the user in the additional information input process described later.

The flower state data item 186 includes the flower state of the guide object indicated by the record as additional information input by the user in the additional information input process described later.

The data item 187 for the number of petals includes the number of petals of the guide object indicated by the record as additional information input by the user in the additional information input process described later.

The leaf shape data item 188 includes the leaf shape of the guide object indicated by the record as additional information input by the user in the additional information input process described later.

For example, the record 189 is a record indicating a guidance object in which the data item 181 of the guidance object identification information is "N1", the object ID is "T1", and the identification information of the image data is "G7872". The color is red, the shape is multiple, the state of the flower is in full bloom, the number of petals is 5 or more, and the shape of the leaf is round.

FIG. 9 shows, for example, a map of a park in which an object included in the object database 150 is arranged. Blocks B1 to B8 are blocks (flower beds) in which plants such as roses, which are objects, are planted. The area around blocks B1 to B8 is a promenade W, and a user having a mobile terminal 200 who is a visitor to the park moves on the promenade W.

FIG. 10 shows a map of the park displaying the lines stored in the line database 140. The line is a line (route, route) that is assumed to be passed by a user who owns the mobile terminal 200 when viewing a plant in a park, and is set in advance, and as shown in FIG. 10, blocks B1 to B8. It is set along the perimeter. Further, a registration point in the object database 150 is set on the line. That is, on the line, there are photographing spots suitable for photographing the plants planted in the blocks B1 to B8.

In FIG. 10, the line is shown as a thick line on the map. For example, the line L1 is a line that continues from the upper left corner of the block B1 to the upper right corner of the block B1. The line L2 is a line that continues from the lower right corner of the block B4 to the upper right corner of the block B4. That is, for example, four lines are set around a rectangular block (flowerbed) so as to correspond to each side.

FIG. 11 shows a map of the park displaying the lines stored in the line database 140, the objects stored in the object database 150, and the registration points. The objects are arranged within the block and are indicated by white circles in FIG. For example, the object T1 is the object shown in record 159 of FIG. The registration point is associated with the object and is set, for example, around the block. In FIG. 11, it is indicated by a black circle. This is because it is assumed that the user moves along the periphery of the blocks B1 to B8 in order to image the plants planted in the blocks B1 to B8 which are the objects. For example, the registration point R1 is associated with the object T1, and the registration point R1 is a registration point set on the line L10. The registration point may be associated with the object and is not limited to the periphery of the block.

FIG. 12 shows an activation screen 400 displayed on the display unit 223 when the additional information registration program 231 stored in the storage unit 230 of the mobile terminal 200 is activated. On the startup screen 400, a camera shooting screen button 401, a text / photo display screen button 402, a map display screen button 403, and various setting buttons 404 are arranged.

A camera shooting screen button 401 for transitioning to the camera shooting screen 410 is arranged at the upper part of the startup screen 400.

A text / photo display screen button 402 for transitioning to the text / photo display screen 420 is arranged next to the camera shooting screen button.

A map display screen button 403 for transitioning to the map display screen 430 is arranged next to the text / photo display screen button.

Various setting buttons 404 for transitioning to various setting screens (not shown) are arranged at the bottom of the startup screen 400. On the various setting screens, it is possible to set whether or not to output the voice, switch the voice between men and women, and set the output language.

FIG. 13 shows a camera shooting screen 410A displayed on the display unit 223 of the mobile terminal 200. The camera shooting screen 410A is a screen when a subject is imaged by the imaging unit 225. A button 412A for capturing an image of a subject displayed in the image display area 411A is arranged at the bottom of the screen.

FIG. 14 shows the screen 410B after the imaging of the subject is completed. A button 412B for analyzing the image of the subject displayed in the captured image display area 411B is arranged at the lower part of the screen. The analysis result is displayed on the text / photo display screen 420.

FIG. 15 shows a text / photo display screen 420 displayed on the display unit 223 of the mobile terminal 200. This is a screen when guidance information such as texts and image examples related to the guidance object generated from the analysis result of the image of the subject captured by the image pickup unit 225 of the mobile terminal 200 is displayed. That is, FIG. 15 is a screen on which guidance information extracted by analyzing an image of a plant captured as a subject by the imaging unit 225 of the mobile terminal 200 with reference to the object database 150 is displayed.

As shown in FIG. 15, the name of the guide object is displayed in the text display area 421. An image of the guide object is displayed in the photo display area 422. The guidance of the text display area 421 may be output as voice from the voice output unit 227 of the mobile terminal 200.

A voice recognition button 423 for executing natural language dialogue processing is arranged at the bottom of the text / photo display screen 420. For example, as shown in FIG. 16, the voice recognition result input by the user is displayed in the voice recognition result display area 425, and for example, as shown in FIG. 17, the result of the natural language dialogue processing is displayed in the text display area 421. ..

Further, an additional information registration button 424 for executing the additional information registration process described later is arranged at the upper right of the text / photo display screen 420. FIG. 18 is an additional information input screen 430A that asks, for example, what color the captured subject is, and prompts the user to input additional information. In the question display area 431A, a question regarding the guide object is displayed. In the option display area 432A, options for the user to select a predetermined answer to the question content displayed in the question display area 431A are displayed.

FIG. 19 shows an additional information input screen 430B in which the last question is displayed in the question display area 431B.

The contents answered by the user in FIGS. 18 and 19 are registered in the re-learning database 180 and are referred to, for example, when analyzing the images captured from the next time onward. With such a configuration, the additional information registration system 10 can easily acquire a wide variety of additional information required for image analysis from the user. As a result, it becomes possible to improve the accuracy of image analysis.

FIG. 20 shows a map display screen 440 displayed on the display unit 223 of the mobile terminal 200. On the map display screen 440, the positions of the subjects already captured are displayed on a map or an aerial photograph. An enlargement button 441 for enlarging and displaying the map by one step is arranged in the upper left upper part of the map display screen 440. A reduction button 442 for reducing and displaying the map by one step is arranged next to the enlargement button 441. A position display button 443 for switching the display / non-display of the current position icon 445, which will be described later, is arranged next to the reduction button 442. The current position icon 445 indicates a position updated by the route matching process described later on the map displayed on the map display screen 440. The guide object icon 444 indicates the position of the already specified guide object on the map displayed on the map display screen 440.

FIG. 21 is a flowchart showing the additional information registration process in the first embodiment. The flow of the additional information registration process in the first embodiment will be described with reference to FIG. 21. In the additional information registration process, for example, when a user who owns the mobile terminal 200 images a plant as a subject in a predetermined place such as a predetermined park, guidance information related to the plant is displayed on the display unit of the mobile terminal 200. By displaying the information, the information is provided to the user, and additional information is input to the user.

First, when the additional information registration program 231 stored in the storage unit 230 of the mobile terminal 200 is activated, the activation screen 400 shown in FIG. 12 is displayed. After that, when the camera shooting screen button 401 in the startup screen 400 is pressed, the transition to the camera shooting screen 410A as shown in FIG. 13, for example, enables the imaging unit 225 to image the subject. Also, swiping to the left returns to the previous screen. For example, when the camera shooting screen button 401 displayed on the startup screen 400 is pressed to transition to the camera shooting screen 410A and then swipe left, the screen returns to the startup screen 400.

In step S101, the CPU 210 takes an image of a predetermined subject and acquires an image. In the first embodiment, for example, a plant planted in a park is taken as an example of the subject, but it may be a facility such as a playset in the park, an animal in a zoo, or the like, and is not particularly limited.

In step S102, the CPU 210 transmits the acquired image to the server 100. Then, the server 100 receives the captured image of the subject from the mobile terminal 200.

In step S103, the CPU 210 detects the imaging position when the subject is imaged by the position information calculation unit 224.

In step S104, the CPU 210 transmits an imaging position information message to the server 100. Then, the server 100 receives the imaging position information message from the mobile terminal 200. The image pickup position information message includes information on the image pickup position when the subject is imaged.

In step S105, the CPU 110 executes a route collation process described later.

In step S106, the CPU 110 executes a guide object search process described later.

In step S107, the CPU 110 transmits a guide object message to the mobile terminal 100. Then, the mobile terminal 200 receives the guidance object message from the server 100. When the mobile terminal 200 receives the guidance object message, the text or image example related to the guidance object may be automatically transitioned to the screen displayed on the display unit 223. In addition, the guidance object message includes information on the guidance object such as the name of a flower. Here, the guide object refers to the object when the subject captured in the guide object search process described later and the object registered in the object database 150 in advance match.

In step S108, the CPU 210 executes an additional information input process described later.

In step S109, the CPU 210 sends an additional information message to the server 100. Then, the server 100 receives the additional information message from the mobile terminal 200. The additional information message includes, for example, an answer input by a user who has imaged the subject in the additional information input process described later, that is, additional information.

In step S110, the CPU 110 executes a classification label registration process described later.

FIG. 22 is a flowchart showing a route matching process executed by the CPU 110. The route matching process will be described with reference to FIG. 22. The route matching process is a process of identifying the registration point closest to the image pickup position based on the image pickup position information message transmitted from the mobile terminal 200. The route collation process is a part of the additional information registration process described above.

In step S201, the CPU 110 determines whether or not the imaging position is closer than the predetermined distance threshold value from the predetermined registration point. The registration point is associated with the object and is set, for example, around the block. In the first embodiment, the predetermined distance threshold value is set to 6 m, but the present invention is not limited to this. In step S201, if the imaging position is closer than the predetermined distance threshold value from the predetermined registration point (step S201 → Y), the CPU 110 proceeds to step S202, and if not (step S201 → N), the CPU 110 performs route matching processing and additional information registration. End the process. That is, in step S201, it is possible to determine whether or not there is a user in, for example, a park or a botanical garden where the object is arranged. Then, when it is determined that the user is not in the park, the CPU 110 ends the route collation process and the additional information registration process.

In step S202, the CPU 110 executes the processes of steps S203 to S206 for each of all the records stored in the line database 140.

In step S203, the CPU 110 determines whether or not the moving direction of the user coincides with the direction from the start point to the end point of the line. In step S203, the CPU 110 proceeds to step S204 if the movement direction of the user matches the direction from the start point to the end point of the line (step S203 → Y), and proceeds to step S207 if they do not match (step S203 → N). .. As a result, it is possible to reduce the choices of the target line when executing the processes of steps S204 to S206.

In step S204, the CPU 110 calculates a perpendicular foot (FIG. 23) on the line with respect to the imaging position. That is, the distance between the imaging position and the line is calculated.

Here, FIG. 23 is a diagram for explaining the foot of the perpendicular line on the line with respect to the imaging position. This will be described with reference to the above-mentioned line L10. The line L10 is a line starting from the registration point R1 and ending at the registration point R3. Point F20 is the imaging position. The point F21 is a foot of a perpendicular line drawn from the point F20 to the line L10 (a line segment connecting the points R1 and R3), and is the point closest to the point F20 on the line L10.

In step S205, the CPU 110 determines whether or not the distance between the imaging position and the foot of the perpendicular line (distance between the point F20 and the point F21 in FIG. 23) is the shortest and is equal to or less than a predetermined distance threshold value. In the first embodiment, the predetermined distance threshold value is set to 4 m from the accuracy of the position information calculation unit 224 of the mobile terminal 200, but the present invention is not limited to this. In step S205, if the distance between the imaging position and the foot of the perpendicular line (distance between the point F20 and the point F21 in FIG. 23) is the shortest and is equal to or less than a predetermined distance threshold value (step S205 → Y), the CPU 110 proceeds to step S206. .. Further, in step S205, if the distance between the imaging position and the foot of the perpendicular line is not the shortest or exceeds a predetermined distance threshold value (step S205 → N), the CPU 110 proceeds to step S207. That is, in step S205, it is an object to specify the line when the distance is the shortest, based on the distance between the imaging position and the line calculated in step S204.

In step S206, the CPU 110 identifies the registration point closest to the imaging position among the registration points set on the line determined in step S205 that the distance is the shortest and is equal to or less than the predetermined distance threshold value.

In step S207, the CPU 110 proceeds to step S208 if the processes of steps S203 to S206 are executed for all the records stored in the line database 140. If there is an unprocessed record, the processes of steps S203 to S206 are executed.

In step S208, the CPU 110 determines whether or not the nearest registration point has been identified. That is, it is determined whether or not the process of step S206 has been executed. When the nearest registration point is specified in step S208 (step S208 → Y), the CPU 110 returns to the additional information registration process. If the nearest registration point is not specified in step S208 (step S208 → N), the CPU 110 ends the route matching process and the additional information registration process.

FIG. 24 is a flowchart showing a guide object search process executed by the CPU 110. The guide object search process will be described with reference to FIG. 24. In the guide object search process, for example, an image of a subject is classified by a classifier, and whether or not the subject is an object associated with a registration point specified by a route matching process or not is referred to in the object database 150. Is. The guide object search process is a part of the additional information registration process described above.

In step S301, the CPU 110 classifies the image of the subject transmitted from the mobile terminal 200 by using, for example, a classifier. In the first embodiment, images are classified based on a plurality of types of features such as "TYPE (that is, whether or not they are loose)", "COLOR", and "SHAPE", but the present invention is not limited to this. In addition, the accuracy of classification can be improved by performing image processing such as blurring the background and then classifying.

In step S302, the CPU 110 determines whether or not the subject is an object registered in advance in the object database 150. That is, the CPU 110 uses the registration point specified by the route matching process and the image classification result in step S301 to determine whether or not the captured subject is an object associated with the registration point. Refer to the object database 150. In step S302, if the subject is an object (step S302 → Y), the CPU 110 proceeds to step S303. In step S302, if the subject is not an object (step S302 → N), the CPU 110 ends the guide object search process and the additional information registration process.

In step S303, the CPU 110 determines the object as a guide object when it is determined in step S302 that the subject is a pre-registered object.

FIG. 25 is a flowchart showing an additional information input process executed by the CPU 210. The additional information input process will be described with reference to FIG. 25. The additional information input process is a process of displaying the information of the guide object determined in the guide object search process on the display unit 223 of the mobile terminal 200 and accepting the input of the additional information about the guide object. The additional information input process is a part of the above-mentioned additional information registration process.

When the mobile terminal 200 receives the guidance object message transmitted from the server 100, the guidance information of the guidance object is displayed on the text / photo display screen 420. In step S401, the CPU 210 determines whether or not the additional information registration button 424 on the upper right of the text / photo display screen 420 is pressed. That is, it is determined whether or not the additional information registration button is pressed by the user. In step S401, the CPU 210 proceeds to step S402 if the additional information registration button 424 at the upper right of the text / photo display screen 420 is pressed (step S401 → Y), and if not pressed (step S401 → N), the additional information input process. And end the additional information registration process.

In step S402, the CPU 210 causes the display unit 223 to display an additional information input screen on which additional information for registering in association with the guide object can be input.

In step S403, the CPU 210 determines whether or not additional information has been input. That is, it is determined whether or not the user has input a predetermined answer to the question content displayed on the additional information input screen 430A. In the first embodiment, the number of questions is five, but the number is not limited to this. If you want to get more information or a wide variety of information, you can increase the number of questions. The question may be about the information registered in the object database 150. If a question asking about the information registered in the object database 150 is included, the answer to the question is used when determining whether or not the additional information corresponds to the guide object in the classification label registration process described later. It becomes possible to judge.

In step S403, if additional information is input (step S403 → Y), the CPU 210 returns to the additional information registration process, and if not (step S403 → N), the CPU 210 ends the additional information input process and the additional information registration process. do. Even if the answers have not been input to all the questions, if the answers have been input to at least one question, it is considered that the additional information has been input, and the process of returning to the additional information registration process can be performed.

FIG. 26 is a flowchart showing a classification label registration process executed by the CPU 110. The classification label registration process will be described with reference to FIG. 26. A process of determining whether or not the additional information input by the user is the correct information corresponding to the guide object, and if it is determined to be the correct information, registering the additional information as a classification label used at the time of image classification. Is. The classification label registration process is a part of the additional information registration process described above.

In step S501, the CPU 110 determines whether or not the additional information corresponds to the guide object. That is, the object database 150 is referred to in order to determine whether or not the input additional information is the correct information corresponding to the guide object. In step S501, when the additional information corresponds to the guide object (step S501 → Y), the process proceeds to step S502, and when the additional information does not correspond to the guide object (step S501 → N), the classification label registration process and End the additional information registration process.

In step S502, the CPU 110 registers the input additional information as a classification label in a predetermined storage means. Here, the classification label is a label for identification to be used for classification processing of, for example, an image, and indicates what the image represents or what is included in the image. The classification label may be given to the image in advance, or may be given based on the information newly input by the user. The classification label is sometimes called "tag" or "tag information". For example, if the image represents the sun, the image is given the classification label "sun". Further, for example, when an image is a book author, the image is given a classification label of "book". Further, the predetermined storage means is, for example, the re-learning database 180, in which information added by the user is registered.

The classification label registered in the re-learning database 180 is referred to, for example, when analyzing the images captured from the next time onward. With such a configuration, the additional information registration system 10 can easily acquire a wide variety of information required for image analysis from the user. As a result, it becomes possible to improve the accuracy of image analysis.

When the voice recognition button 423 located at the bottom of the text / photo display screen 420 is pressed, the natural language dialogue process executed by the CPU 210 is based on the information added to the re-learning database 180. You may create a response to a question entered by the user.

According to the database generation method, the database generation device, and the database generation program of the first embodiment, the number of samples of teacher data is efficiently increased by inputting additional information corresponding to the object to be imaged. At the same time, the degree of subdivision of the classification label can be increased.

In the first embodiment, the subject is a plant planted in the park as a facility, but the subject is not limited to this, and equipment such as playsets in the park, cages in which animals are bred, and insects are frequently used. The spot observed in the subject may be the subject. Further, the facility is not limited to a park, and may be an outdoor facility such as a theme park, an amusement park, or a zoo.

Further, in the first embodiment, the registration point closest to the imaging position is specified in the route matching process, but a plurality of registration points including the registration point second closest to the imaging position may be specified. .. Then, when there are a plurality of registered points, the object database 150 may be referred to to determine which registered point the subject is associated with.

Further, in the first embodiment, the additional information input screen is displayed when the additional information registration button 424 on the upper right of the text / photo display screen 420 is pressed in the additional information input process. May be configured to automatically transition to the additional information input screen when the mobile terminal 200 receives.

(Embodiment 2)
In the first embodiment, the additional information registration system 10 is configured to include the mobile terminal 200 and the server 100. In the second embodiment, the server 100 is not provided, and only the mobile terminal 200 is configured.

FIG. 27 is a diagram showing a schematic configuration of the mobile terminal 200B according to the second embodiment. Compared with the mobile terminal 200 in the first embodiment, there is no communication unit 228, and the CPU 210 executes the route matching process, the guide object search process, and the classification label registration process executed by the CPU 110 of the server 100 in the first embodiment. That is, the CPU 210 functions as an acquisition means, a position detection means, a first determination means, a second determination means, a display means, and a registration means by executing the additional information registration process described later. Further, the line database 140, the object database 150, the position information database 170, and the relearning database 180 are stored in the storage unit 230.

FIG. 28 is a flowchart showing the additional information registration process in the second embodiment. The flow of the additional information registration process in the second embodiment will be described with reference to FIG. 28.

In step S601, the CPU 210 takes an image of a predetermined subject and acquires an image.

In step S602, the CPU 210 detects the imaging position when the subject is imaged by the position information calculation unit 224.

In step S603, the CPU 210 executes the route collation process executed by the CPU 110 of the server 100 in the first embodiment.

In step S604, the CPU 210 executes the guide object search process executed by the CPU 110 of the server 100 in the first embodiment.

In step S605, the CPU 210 executes the above-mentioned additional information input process.

In step S606, the CPU 210 executes the classification label registration process executed by the CPU 110 of the server 100 in the first embodiment.

As described above, in the second embodiment, the additional information registration system 10 can be configured only by the mobile terminal 20 without the server 100.

According to the database generation method, the database generation device, and the database generation program of the second embodiment, the number of samples of teacher data is efficiently increased by inputting additional information corresponding to the object to be imaged. At the same time, the degree of subdivision of the classification label can be increased.

In the second embodiment, the subject is a plant planted in the park as a facility, but the subject is not limited to this, and equipment such as playsets in the park, cages in which animals are bred, and insects are frequently used. The spot observed in the subject may be the subject. Further, the facility is not limited to a park, and may be an outdoor facility such as a theme park, an amusement park, or a zoo.

Although some embodiments of the present invention have been described above, these embodiments are merely examples and do not limit the technical scope of the present invention. The present invention can take various other embodiments, and various modifications such as omission and substitution can be made without departing from the gist of the present invention. These embodiments are included in the scope and gist of the invention described in the present specification and the like, and are also included in the scope of the invention described in the claims and the equivalent scope thereof.

The inventions described in the claims originally attached to the application of this application are added below. The claim numbers given in the appendix are the scope of the claims originally attached to the application for this application.

[Appendix 1]
A position detection step of detecting an imaging position when the subject is imaged in order to acquire an image of a predetermined subject, and a position detection step.
A first determination step for determining whether or not the imaging position detected by the position detection step is closer than a predetermined distance threshold value from a predetermined registration point, and
When it is determined by the first determination step that the imaging position is closer than the predetermined distance threshold value from the predetermined registration point, the second determination as to whether or not the subject is a pre-registered object. Steps and
When the subject is determined to be the object by the second determination step, the display step of displaying a screen on which the user can input additional information for registering the subject in association with the image. ,
When the additional information is input to the screen displayed by the display step, the registration step of registering the input additional information as a classification label in a predetermined storage means, and
A database generation method characterized by having.
[Appendix 2]
An acquisition step of acquiring an image of a predetermined subject by imaging the subject, and
A position detection step for detecting an imaging position when the subject is imaged by the acquisition step, and a position detection step.
A first determination step for determining whether or not the imaging position detected by the position detection step is closer than a predetermined distance threshold value from a predetermined registration point, and
When it is determined by the first determination step that the imaging position is closer than the predetermined distance threshold value from the predetermined registration point, the second determination as to whether or not the subject is a pre-registered object. Steps and
When the subject is determined to be the object by the second determination step, the user can input additional information for registering in association with the image acquired by the acquisition step. Display steps to display the screen and
When the additional information is input to the screen displayed by the display step, the registration step of registering the input additional information as a classification label in a predetermined storage means, and
A database generation method characterized by having.
[Appendix 3]
The database generation method according to Appendix 1 or 2, wherein the registration point is associated with the object.
[Appendix 4]
The registration point is set on a predetermined line that is assumed to pass when the user moves.
The first determination step is characterized in that it determines whether or not the imaging position is closer than a predetermined distance threshold value from the registration point based on the distance between the imaging position and the line. The database generation method according to any one of the items.
[Appendix 5]
It has a direction detection step for detecting the direction in which the user moves.
In the first determination step, when the moving direction detected by the direction detection step and the direction from the start point to the end point of the line coincide with each other, the imaging position is based on the distance between the imaging position and the line. The database generation method according to Appendix 4, wherein is determined whether or not is closer than a predetermined distance threshold value from the registration point.
[Appendix 6]
A position detecting means for detecting an imaging position when the subject is imaged in order to acquire an image of a predetermined subject, and
A first determining means for determining whether or not the imaging position detected by the position detecting means is closer than a predetermined distance threshold value from a predetermined registration point, and
When the first determination means determines that the imaging position is closer than the predetermined distance threshold value from the predetermined registration point, the second determination as to whether or not the subject is a pre-registered object. Means and
When the subject is determined to be the object by the second determination means, the display control means for displaying a screen on which the user can input additional information for registering the subject in association with the image. When,
When the additional information is input to the screen displayed by the display control means, the registration means for registering the input additional information as a classification label in a predetermined storage means, and
A database generator characterized by comprising.
[Appendix 7]
An acquisition means for acquiring an image of a predetermined subject by imaging the subject, and
A position detecting means for detecting an imaging position when the subject is imaged by the acquiring means, and a position detecting means.
A first determining means for determining whether or not the imaging position detected by the position detecting means is closer than a predetermined distance threshold value from a predetermined registration point, and
When the first determination means determines that the imaging position is closer than the predetermined distance threshold value from the predetermined registration point, the second determination as to whether or not the subject is a pre-registered object. Means and
When the subject is determined to be the object by the second determination means, the user can input additional information for registering in association with the image acquired by the acquisition means. Display control means to display the screen and
When the additional information is input to the screen displayed by the display control means, the registration means for registering the input additional information as a classification label in a predetermined storage means, and
A database generator characterized by comprising.
[Appendix 8]
The database generator according to Appendix 6 or 7, wherein the registration point is associated with the object.
[Appendix 9]
Computer,
A position detecting means for detecting an imaging position when the subject is imaged in order to acquire an image of a predetermined subject.
A first determining means for determining whether or not the imaging position detected by the position detecting means is closer than a predetermined distance threshold value from a predetermined registration point.
When the first determination means determines that the imaging position is closer than the predetermined distance threshold value from the predetermined registration point, the second determination as to whether or not the subject is a pre-registered object. means,
When the subject is determined to be the object by the second determination means, the display control means for displaying a screen on which the user can input additional information for registering the subject in association with the image. ,
When the additional information is input to the screen displayed by the display control means, the registration means for registering the input additional information as a classification label in a predetermined storage means.
A database generator to function as.
[Appendix 10]
Computer,
An acquisition means for acquiring an image of a predetermined subject by imaging the subject,
A position detecting means for detecting an imaging position when the subject is imaged by the acquiring means,
A first determining means for determining whether or not the imaging position detected by the position detecting means is closer than a predetermined distance threshold value from a predetermined registration point.
When the first determination means determines that the imaging position is closer than the predetermined distance threshold value from the predetermined registration point, the second determination as to whether or not the subject is a pre-registered object. means,
When the subject is determined to be the object by the second determination means, the user can input additional information for registering in association with the image acquired by the acquisition means. Display control means to display the screen,
When the additional information is input to the screen displayed by the display control means, the registration means for registering the input additional information as a classification label in a predetermined storage means.
A database generator to function as.
[Appendix 11]
The database generation program according to Appendix 9 or 10, wherein the registration point is associated with the object.

10 Additional information registration system 100 Server 110 CPU
121 RAM
122 Operation unit 123 Display unit 124 Communication unit 129 Bus 130 Storage unit 140 Line database 150 Object database 170 Location information database 180 Relearning database 200 Mobile terminal 210 CPU
221 RAM
222 Operation unit 223 Display unit 224 Position information calculation unit 225 Imaging unit 226 Audio input unit 227 Audio output unit 228 Communication unit 229 Bus 230 Storage unit 231 Additional information registration program 500 Network

The present invention relates to a database generation method , an additional information registration system, a database generation device and a program .

Therefore, an object of the present invention is to provide a database generation method, an additional information registration system, a database generation device, and a program capable of increasing the degree of subdivision of classification labels while efficiently increasing the number of samples of teacher data. do.

In order to achieve the above object, the database generation method according to the present invention is a database generation method executed by the additional information registration system, and when the subject is imaged in order to acquire an image of a predetermined subject. A position detection step for detecting an imaging position, a first determination step for determining whether or not the imaging position detected by the position detection step is closer than a predetermined distance threshold value from a predetermined registration point, and the first determination step. When the determination step determines that the imaging position is closer than the predetermined distance threshold value from the predetermined registration point, the second determination step of determining whether or not the subject is a pre-registered type of plant. When the subject is determined to be the pre-registered type of plant by the second determination step, the user inputs the state information of the plant as additional information for registering in association with the image. a display step of displaying the possible screen to be, the case where the state information on the screen displayed by the display step is input, registration to register in a predetermined storage unit the state information the input as the classification label It is characterized by having a step and.

The database generation method of another aspect according to the present invention is the directed to a database creation method database generation apparatus executes an acquiring step of acquiring an image of the subject by the imaging for a given subject, by the acquisition step A position detection step for detecting an imaging position when a subject is imaged, and a first determination for determining whether or not the imaging position detected by the position detection step is closer than a predetermined distance threshold from a predetermined registration point. When it is determined by the step and the first determination step that the imaging position is closer than the predetermined distance threshold from the predetermined registration point, it is determined whether or not the subject is a pre-registered type of plant. When the subject is determined to be the pre-registered type of plant by the second determination step and the second determination step, in order to register the subject in association with the image acquired by the acquisition step. the status information of the plant as additional information, a display step of displaying the possible screen the user inputting, when the state information on the screen displayed by the display step is entered, which is the input It is characterized by having a registration step of registering state information as a classification label in a predetermined storage means.

Further, the additional information registration system according to the present invention includes a position detecting means for detecting an imaging position when the subject is imaged in order to acquire an image of a predetermined subject, and the imaging detected by the position detecting means. When the first determination means for determining whether or not the position is closer than the predetermined distance threshold value from the predetermined registration point and the imaging position are closer than the predetermined distance threshold value from the predetermined registration point by the first determination means. When the determination is made, the second determination means for determining whether or not the subject is a pre-registered type of plant and the second determination means for the subject to be a pre-registered type of plant . When it is determined that there is, the state information of the plant is displayed as additional information for registering in association with the image by the display control means for displaying a screen that can be input by the user and the display control means. When the state information is input to the screen , the registration means for registering the input state information as a classification label in a predetermined storage means is provided.

Further, the database generation device according to the present invention includes an acquisition means for acquiring an image of the subject by imaging a predetermined subject, a position detecting means for detecting an imaging position when the subject is imaged by the acquisition means, and a position detecting means. A first determining means for determining whether or not the imaging position detected by the position detecting means is closer than a predetermined distance threshold from a predetermined registration point, and a predetermined registration of the imaging position by the first determining means. When it is determined that the subject is closer than a predetermined distance threshold from the point, the subject is determined by the second determination means for determining whether or not the subject is a plant of a pre-registered type, and the second determination means. When it is determined that the plant is of the type registered in advance , the user may input the state information of the plant as additional information for registering in association with the image acquired by the acquisition means. A display control means for displaying a possible screen, and a registration means for registering the input state information as a classification label in a predetermined storage means when the state information is input to the screen displayed by the display control means. It is characterized by having.

Further, the program according to the present invention detects the computer of the additional information registration system by a position detecting means for detecting an imaging position when the subject is imaged in order to acquire an image of a predetermined subject, and the position detecting means. The first determination means for determining whether or not the imaging position is closer than the predetermined distance threshold from the predetermined registration point, and the first determination means causes the imaging position to be closer to the predetermined distance threshold than the predetermined registration point. When it is determined that the subject is close to the subject, the second determination means for determining whether or not the subject is a plant of the type registered in advance , the second determination means means that the subject is of the type registered in advance. When it is determined that the plant is a plant, the display control means for displaying the screen on which the user can input the state information of the plant as additional information for registering the image in association with the image, the display control means. When the state information is input to the displayed screen , it is characterized in that it functions as a registration means for registering the input state information as a classification label in a predetermined storage means.

Further, in the program of another aspect according to the present invention, the computer of the database generator is an acquisition means for acquiring an image of the subject by imaging a predetermined subject, and an imaging position when the subject is imaged by the acquisition means. By the position detecting means for detecting the above, the first determining means for determining whether or not the imaging position detected by the position detecting means is closer than the predetermined distance threshold from the predetermined registration point, and the first determining means. When it is determined that the imaging position is closer than the predetermined distance threshold from the predetermined registration point, the second determination means for determining whether or not the subject is a pre-registered type of plant, the second determination means. When the determination means determines that the subject is a plant of the type registered in advance , the state information of the plant is added as additional information for registering in association with the image acquired by the acquisition means. display control means for a user to display a screen capable of inputting, the display control if the status information to the screen displayed is input by means predetermined storage unit the state information the input as the classification label It is characterized in that it functions as a registration means for registering in.

Claims (11)

A database generation method executed by the additional information registration system.
A position detection step of detecting an imaging position when the subject is imaged in order to acquire an image of a predetermined subject, and a position detection step.
A first determination step for determining whether or not the imaging position detected by the position detection step is closer than a predetermined distance threshold value from a predetermined registration point, and
When it is determined by the first determination step that the imaging position is closer than the predetermined distance threshold value from the predetermined registration point, the subject is an object registered in advance corresponding to the predetermined registration point. The second determination step to determine whether or not
When the subject is determined to be the object by the second determination step, the user can input additional information for registering in association with the image as answer information for a predetermined question. Display steps to display a question screen and
When the answer information is input to the question screen displayed by the display step, a registration step of registering the input answer information as a classification label in a predetermined storage means, and
A database generation method characterized by having. A database generation method executed by the additional information registration system.
An acquisition step of acquiring an image of a predetermined subject by imaging the subject, and
A position detection step for detecting an imaging position when the subject is imaged by the acquisition step, and a position detection step.
A first determination step for determining whether or not the imaging position detected by the position detection step is closer than a predetermined distance threshold value from a predetermined registration point, and
When it is determined by the first determination step that the imaging position is closer than the predetermined distance threshold value from the predetermined registration point, the subject is an object registered in advance corresponding to the predetermined registration point. The second determination step to determine whether or not
When the subject is determined to be the object by the second determination step, additional information for registering the subject in association with the image acquired by the acquisition step is used as answer information for a predetermined question. A display step that displays a question screen that can be entered by the user,
When the answer information is input to the question screen displayed by the display step, a registration step of registering the input answer information as a classification label in a predetermined storage means, and
A database generation method characterized by having. The registration points are associated with each object.
The database generation method according to claim 1 or 2, wherein the database is generated. The registration point is set on a predetermined line that is assumed to pass when the user moves.
The first determination step determines whether or not the imaging position is closer than a predetermined distance threshold value from the registration point based on the distance between the imaging position and the line.
The database generation method according to any one of claims 1 to 3, wherein the database is generated. It has a direction detection step for detecting the direction in which the user moves.
In the first determination step, when the moving direction detected by the direction detection step and the direction from the start point to the end point of the line coincide with each other, the imaging position is based on the distance between the imaging position and the line. Determines whether or not is closer than the predetermined distance threshold from the registration point.
The database generation method according to claim 4, wherein the database is generated. A position detecting means for detecting an imaging position when the subject is imaged in order to acquire an image of a predetermined subject, and
A first determining means for determining whether or not the imaging position detected by the position detecting means is closer than a predetermined distance threshold value from a predetermined registration point, and
When the first determination means determines that the imaging position is closer than the predetermined distance threshold value from the predetermined registration point, the subject is an object registered in advance corresponding to the predetermined registration point. A second determination means for determining whether or not,
When the subject is determined to be the object by the second determination means, the user can input additional information for registering in association with the image as answer information for a predetermined question. Display control means to display a question screen and
When the answer information is input to the question screen displayed by the display control means, the registration means for registering the input answer information as a classification label in a predetermined storage means, and
A database generator characterized by comprising. An acquisition means for acquiring an image of a predetermined subject by imaging the subject, and
A position detecting means for detecting an imaging position when the subject is imaged by the acquiring means, and a position detecting means.
A first determining means for determining whether or not the imaging position detected by the position detecting means is closer than a predetermined distance threshold value from a predetermined registration point, and
When the first determination means determines that the imaging position is closer than the predetermined distance threshold value from the predetermined registration point, the subject is an object registered in advance corresponding to the predetermined registration point. A second determination means for determining whether or not,
When the subject is determined to be the object by the second determination means, additional information for registering the subject in association with the image acquired by the acquisition means is used as answer information for a predetermined question. Display control means to display a question screen that can be entered by the user,
When the answer information is input to the question screen displayed by the display control means, the registration means for registering the input answer information as a classification label in a predetermined storage means, and
A database generator characterized by comprising. The registration points are associated with each object.
The database generator according to claim 6 or 7. Computer,
A position detecting means for detecting an imaging position when the subject is imaged in order to acquire an image of a predetermined subject.
A first determining means for determining whether or not the imaging position detected by the position detecting means is closer than a predetermined distance threshold value from a predetermined registration point.
When the first determination means determines that the imaging position is closer than the predetermined distance threshold value from the predetermined registration point, the subject is an object registered in advance corresponding to the predetermined registration point. Second determination means for determining whether or not,
When the subject is determined to be the object by the second determination means, the user can input additional information for registering in association with the image as answer information for a predetermined question. Display control means to display a question screen,
When the answer information is input to the question screen displayed by the display control means, the registration means for registering the input answer information as a classification label in a predetermined storage means.
A database generator to function as. Computer,
An acquisition means for acquiring an image of a predetermined subject by imaging the subject,
A position detecting means for detecting an imaging position when the subject is imaged by the acquiring means,
A first determining means for determining whether or not the imaging position detected by the position detecting means is closer than a predetermined distance threshold value from a predetermined registration point.
When the first determination means determines that the imaging position is closer than the predetermined distance threshold value from the predetermined registration point, the subject is an object registered in advance corresponding to the predetermined registration point. Second determination means for determining whether or not,
When the subject is determined to be the object by the second determination means, additional information for registering the subject in association with the image acquired by the acquisition means is used as answer information for a predetermined question. Display control means to display a question screen that can be entered by the user,
When the answer information is input to the question screen displayed by the display control means, the registration means for registering the input answer information as a classification label in a predetermined storage means.
A database generator to function as. The registration points are associated with each object.
The database generation program according to claim 9 or 10.

Images