JP7315192B2 - Processing device, processing method and program - Google Patents

Description

The present invention relates to a processing device, processing method and program.

Patent Literature 1 discloses a technique of collecting an image of a payment slip brought by a customer, determining the type of payment slip from the collected image, recognizing characters written on the payment slip, and displaying fees and arrears on a screen for the customer.

Patent Document 2 discloses a technique of registering an item image for each item in advance and confirming whether the ordered item has been correctly put out based on the photographed image of the item and the item image when the ordered item is put out.

JP 2015-5156 A Japanese Patent Application Laid-Open No. 2004-206177

In stores such as convenience stores, business is diversifying, and the knowledge required of store clerks is increasing. Therefore, the quality of service provided by each store clerk may vary depending on the knowledge level.

SUMMARY OF THE INVENTION An object of the present invention is to reduce the inconvenience caused by variations in the quality of services provided depending on the knowledge of store clerks.

According to the invention,
recognition means for recognizing a business object included in an image;
Acquisition means for acquiring business support information for supporting the business of the store clerk regarding the recognized business object;
output means for outputting the acquired business support information;
is provided.

Moreover, according to the present invention,
the computer
a recognition step of recognizing business objects included in the image;
an obtaining step of obtaining, from a storage means, business support information for supporting the business of the store clerk regarding the recognized business object;
an output step of outputting the acquired business support information;
A processing method is provided for performing

Moreover, according to the present invention,
the computer,
recognition means for recognizing business objects included in the image;
Acquisition means for acquiring, from a storage means, business support information for supporting the business of the store clerk regarding the recognized business object;
output means for outputting the acquired business support information;
A program is provided to act as a

ADVANTAGE OF THE INVENTION According to this invention, the inconvenience that the quality of the service provided depends on a salesclerk's knowledge can be reduced.

It is a figure which shows an example of the functional block diagram of the system of this embodiment. It is a figure which shows an example of the hardware constitutions of the server of this embodiment, and a terminal device. It is a figure which shows an example of the functional block diagram of the processing apparatus of this embodiment. It is a figure which shows typically an example of the merchandise information of this embodiment. 4 is a diagram schematically showing an example of slip information according to the embodiment; FIG. It is a figure which shows typically an example of the information memorize|stored in the business support information storage part of this embodiment. It is a figure which shows typically an example of the information memorize|stored in the business support information storage part of this embodiment. It is a flow chart which shows an example of a flow of a processor of this embodiment. It is a figure which shows an example of the functional block diagram of the system of this embodiment. It is a figure which shows typically an example of the information output by the processing apparatus of this embodiment. It is a figure which shows typically an example of the information output by the processing apparatus of this embodiment. It is a figure which shows typically an example of the information output by the processing apparatus of this embodiment.

<First Embodiment>
First, an outline of the system of this embodiment will be described. The system of this embodiment is used in stores. The type of store that uses the system of this embodiment is not limited, and the system can be used in any store such as convenience stores, supermarkets, home appliance mass retailers, general stores, cosmetics stores, sporting goods stores, and the like.

FIG. 1 shows a functional block diagram of the system of this embodiment. As illustrated, the system of this embodiment has a server 1 and a terminal device 2 . The server 1 and the terminal device 2 can communicate with each other by wire and/or wirelessly. In this embodiment, the server 1 has the function of the processing device 10 .

The terminal device 2 is a terminal operated by a store clerk. Examples of the terminal device 2 include a tablet terminal, a smart phone, a POS (point of sales) register, and the like, but other devices may be used. The terminal device 2 has a camera function. When the store clerk operates the terminal device 2, the terminal device 2 takes an image of an arbitrary business object. The terminal device 2 then transmits the generated image data to the server 1 .

Note that the terminal device 2 may have a function of recognizing the business object included in the image by image analysis of the generated image data. In this case, the terminal device 2 may transmit the recognition result of the business object to the server 1 instead of the image data.

The server 1 (processing device 10) recognizes the business object included in the image. The server 1 may receive image data from the terminal device 2 and analyze the image data to recognize the business object included in the image. In addition, the server 1 may receive the business object recognition result from the terminal device 2 .

Next, the server 1 acquires business support information that supports the salesclerk's business regarding the recognized business object. The server 1 then transmits the acquired business support information to the terminal device 2 . The terminal device 2 presents the business support information received from the server 1 to the store clerk.

According to the system of the present embodiment, a salesclerk can acquire business support information about a business object by a simple operation of operating the terminal device 2 and photographing the business object for which the business content is desired.

Next, an example of hardware configurations of the server 1 and the terminal device 2 will be described. The functions provided by the server 1 and the terminal device 2 are realized by an arbitrary combination of hardware and software centering on the CPU (Central Processing Unit) of any computer, memory, a program loaded into the memory, a storage unit such as a hard disk that stores the program (in addition to the program stored in advance from the stage of shipping the device, a storage medium such as a CD (Compact Disc) or a program downloaded from a server on the Internet can also be stored), and an interface for network connection. It should be understood by those skilled in the art that there are various modifications to the implementation method and apparatus.

FIG. 2 is a block diagram illustrating the hardware configuration of each of the server 1 and the terminal device 2 of this embodiment. As shown in FIG. 2, each of the server 1 and the terminal device 2 has a processor 1A, a memory 2A, an input/output interface 3A, a peripheral circuit 4A and a bus 5A. The peripheral circuit 4A includes various modules. Note that the peripheral circuit 4A may not be provided.

The bus 5A is a data transmission path for mutually transmitting and receiving data between the processor 1A, memory 2A, peripheral circuit 4A and input/output interface 3A. The processor 1A is, for example, an arithmetic processing device such as a CPU (Central Processing Unit) or a GPU (Graphics Processing Unit). The memory 2A is, for example, a RAM (Random Access Memory) or a ROM (Read Only Memory). The input/output interface 3A includes an input device (e.g. keyboard, mouse, microphone, etc.), an interface for acquiring information from an external device, an external server, an external sensor, etc., an output device (e.g., display, speaker, printer, mailer, etc.), an interface for outputting information to an external device, an external server, etc. The processor 1A can issue commands to each module and perform calculations based on the calculation results thereof.

Next, the functional configuration of the processing device 10 will be described. As described above, in this embodiment, the server 1 has the functions of the processing device 10 .

FIG. 3 shows an example of a functional block diagram of the processing device 10 of this embodiment. As illustrated, the processing device 10 has a recognition section 11 , an acquisition section 12 and an output section 13 .

The recognition unit 11 recognizes business objects included in the image. The recognition unit 11 recognizes the business object included in the image by the following processing example 1 or processing example 2. FIG.

"Processing example 1"
In process example 1, the processing device 10 (server 1 ) receives image data from the terminal device 2 . Then, the recognition unit 11 performs image recognition processing on the image data to recognize business objects included in the image. For example, the recognition unit 11 may use product information (see FIG. 4) in which identification information of a product (business object) and features of the appearance of the product are associated with each other. The recognition unit 11 may recognize the product included in the image data by matching the features included in the acquired image data with the appearance features of the product included in the product information.

In addition, the recognition unit 11 may use slip information (see FIG. 5) in which identification information of a slip (business object) and features of the appearance of the slip are associated with each other. The recognition unit 11 may recognize the slip included in the image data by comparing the features included in the acquired image data with the features of the appearance of the slip included in the slip information. A slip is, for example, a payment slip for utility bills, taxes, and various other payments.

"Processing example 2"
In process example 2, the processing device 10 (server 1 ) receives the result of image recognition processing from the terminal device 2 . Then, the recognition unit 11 acquires the result of the image recognition processing. The result of the image recognition processing includes the identification information of the business object included in the image (eg, product identification information, slip identification information).

In the case of this processing example, the terminal device 2 performs image recognition processing on the image data, and recognizes business objects included in the image. For example, the terminal device 2 may use product information (see FIG. 4) in which the identification information of the product (business object) and the features of the appearance of the product are associated with each other. The terminal device 2 may recognize the product included in the image data by comparing the features included in the acquired image data with the appearance features of the product included in the product information.

In addition, the terminal device 2 may use slip information (see FIG. 5) that associates the identification information of the slip (business object) with the features of the appearance of the slip. The terminal device 2 may recognize the slip included in the image data by comparing the features included in the acquired image data with the features of the appearance of the slip included in the slip information.

Returning to FIG. 3 , the acquisition unit 12 acquires business support information that supports the business of the store clerk regarding the business object recognized by the recognition unit 11 . The acquisition unit 12 acquires business support information corresponding to the business object recognized by the recognition unit 11 from the business support information storage unit that stores identification information of the business object and business support information in association with each other. 6 and 7 schematically show an example of information stored in the business support information storage unit. Details of the business support information will be described in the following embodiments.

The processing device 10 may have the business support information storage unit, or another device configured to be able to communicate with the processing device 10 may have the business support information storage unit.

Returning to FIG. 3 , the output unit 13 outputs the business support information acquired by the acquisition unit 12 . The output unit 13 of this embodiment transmits the business support information to the terminal device 2 .

Next, an example of the processing flow of the processing device 10 will be described with reference to the flowchart of FIG.

In S10, the recognition unit 11 recognizes the business object included in the image. For example, the recognition unit 11 performs image recognition processing on image data transmitted from the terminal device 2, and recognizes business objects included in the image. Alternatively, the recognition unit 11 acquires the result of image recognition processing transmitted from the terminal device 2 .

In S11, the acquisition unit 12 acquires business support information that supports the business of the store clerk regarding the business object recognized in S10. The acquisition unit 12 acquires the business support information corresponding to the business object recognized in S11 from the business support information storage unit that stores the identification information of the business object and the business support information in association with each other.

In S12, the output unit 13 outputs the acquired business support information. The output unit 13 transmits the business support information to the terminal device 2 . The terminal device presents the received business support information to the operator (clerk). For example, the terminal device displays business support information on the display.

According to the processing device 10 of the present embodiment described above, it is possible to recognize a business object included in an image taken by a salesclerk and present business support information to the salesclerk to support the business of the salesclerk regarding the business object. A store clerk can acquire business support information about a business object by a simple operation of operating the terminal device 2 and photographing the business object for which the business content is desired.

Since the operation is easy, the store clerk can easily and quickly use the function in any situation to obtain various types of business support information. A store clerk who does not store various business support information can provide the same service as a store clerk who has various business support information as knowledge by using the function.

<Second embodiment>
This embodiment differs from the first embodiment in that the terminal device 2 has the functions of the processing device 10 as shown in FIG. Differences from the first embodiment will be described below.

The hardware configurations of the server 1 and the terminal device 2 are the same as in the first embodiment. An example of a functional block diagram of the processing device 10 is shown in FIG. 3 as in the first embodiment.

The recognition unit 11 performs image recognition processing on the image data generated by its own device, and recognizes business objects included in the image. Details of the image recognition processing are the same as in the first embodiment.

The acquisition unit 12 acquires business support information corresponding to the business object recognized by the recognition unit 11 from the business support information storage unit that stores identification information of the business object and business support information in association with each other. An example of information stored in the business support information storage unit is the same as in the first embodiment.

The terminal device 2 may have the business support information storage unit, or the external device (eg, server 1) may have the business support information storage unit. In the latter case, the acquisition unit 12 may transmit information identifying the business object recognized by the recognition unit 11 to the external device and request the corresponding business support information. Then, the acquisition unit 12 may acquire the business support information transmitted from the external device.

The output unit 13 outputs the business support information acquired by the acquisition unit 12 . The output unit 13 may display the business support information on a display, may project the business support information at an arbitrary position with a projection device, or may print the business support information with a printer. If the terminal device 2 is an electronic cash register, the output unit 13 may display the business support information on a display facing the store clerk.

An example of the processing flow of the processing device 10 of this embodiment is the same as that of the first embodiment.

According to the processing apparatus 10 of this embodiment, the same effects as those of the first embodiment can be achieved.

<Third Embodiment>
This embodiment differs from the first and second embodiments in that business support information is materialized. Differences from the first and second embodiments will be described below. In this embodiment, the server 1 may have the function of the processing device 10 (similar to the first embodiment), and the terminal device 2 may have the function of the processing device 10 (similar to the second embodiment).

The hardware configurations of the server 1 and the terminal device 2 are the same as in the first and second embodiments. An example of a functional block diagram of the processing device 10 is shown in FIG. 3, as in the first and second embodiments.

The recognition unit 11 recognizes products as business objects. The business support information acquired by the acquisition unit 12 includes information about products. The information about the product may include information specific to the store, such as the display location of the product, the inventory status of the product (presence or absence of inventory, the number of products in stock, etc.), the scheduled arrival date of the product, the selling price of the product, and the like. The product inventory status stored in the business support information storage unit may be updated as needed based on information from the POS system. In addition, the business support information may include product-specific information such as product specifications, place of origin, place of manufacture, raw materials, and the like.

FIG. 10 shows an example of business support information output by the output unit 13. As shown in FIG. The business support information includes identification information (product name) of the product recognized by image recognition processing, the display location of the product, the inventory status of the product, and the scheduled arrival date of the product.

For example, a customer who brings a product package or a customer who has a product package displayed at a store may show the product package to a store clerk and ask about the display location, stock status, arrival schedule, sales price, product specifications, place of origin, place of manufacture, raw materials, etc. In such a case, the store clerk operates the terminal device 2 to photograph the product package, thereby obtaining the business support information of the product as shown in FIG. Then, the store clerk can reply to the customer based on the business support information.

FIG. 11 shows another example of business support information output by the output unit 13. As shown in FIG. Similar to the business support information in FIG. 10, the business support information includes identification information (product name) of the product recognized by image recognition processing, the display location of the product, the inventory status of the product, and the scheduled arrival date of the product. It should be noted that the display location differs from the business support information in FIG. 10 in that the display location is the shelf installed at the checkout counter and the position within the shelf.

Cigarettes, for example, may be displayed on shelves installed at checkout counters. In this case, the customer tells the clerk which brand he wishes to purchase. Then, the store clerk takes out the brand from the shelf. If there are many brands of cigarettes on display, it may be difficult to find the advertised brand.

Therefore, when a customer who brings a merchandise package expresses a desire to purchase the brand, the store clerk operates the terminal device 2 to photograph the merchandise package, thereby obtaining business support information of the merchandise as shown in FIG. Then, the store clerk can quickly grasp the display position of the product based on the business support information.

An example of the processing flow of the processing apparatus 10 of this embodiment is the same as in the first and second embodiments.

According to the processing apparatus 10 of this embodiment, the same effects as those of the first and second embodiments can be realized. In addition, according to the processing device 10 of the present embodiment, the store clerk can easily and quickly acquire various types of information about the product by a simple operation of photographing the product. As a result, it is possible to promptly respond to customers (answer questions, register processing, etc.).

Here, a modified example of this embodiment will be described. The product information (see FIG. 4) of the present embodiment may associate product identification information with current and past package appearance features of the product. Then, the entity that performs the image recognition process may recognize the business object included in the image based on the product information. According to the modification, the above functions can be realized even when the product package is changed.

<Fourth Embodiment>
This embodiment differs from the first and second embodiments in that business support information is materialized. Differences from the first and second embodiments will be described below. In this embodiment, the server 1 may have the function of the processing device 10 (similar to the first embodiment), and the terminal device 2 may have the function of the processing device 10 (similar to the second embodiment). Further, the business support information of this embodiment may or may not have the business support information described in the third embodiment.

The hardware configurations of the server 1 and the terminal device 2 are the same as in the first to third embodiments. An example of a functional block diagram of the processing device 10 is shown in FIG. 3, as in the first to third embodiments.

The recognition unit 11 recognizes a slip as a business object. A slip is, for example, a payment slip for utility bills, taxes, and various other payments. The business support information acquired by the acquiring unit 12 includes information indicating the work to be done for each slip in the receipt agent business using the slip. For example, the business support information may include information indicating a barcode written on each slip that needs to be read in the collection agent business. In addition, the business support information may include information indicating the position of the receipt stamp in the slip.

FIG. 12 shows an example of business support information output by the output unit 13. As shown in FIG. The business support information indicates the slip identification information recognized by the image recognition process, the barcode to be read, and the position to press the receipt stamp. Also, the business support information shows a part to be separated and handed over to the customer. Specifically, a sample slip is displayed, and the bar code to be read on the sample, the position to press the receipt stamp, and the part to be cut off and handed over to the customer are shown. Although information (barcodes, characters, numbers, symbols, etc.) written on the slip is omitted in the illustrated slip sample, the business support information actually output by the output unit 13 may include such information.

An example of the processing flow of the processing apparatus 10 of this embodiment is the same as in the first to third embodiments.

According to the processing apparatus 10 of this embodiment, the same effects as those of the first to third embodiments can be achieved. In addition, according to the processing device 10 of the present embodiment, the store clerk can easily and quickly acquire information indicating the work to be done for each slip in the collection agency business using the slip by a simple operation of photographing the slip. As a result, it becomes possible to quickly respond to customers (receiving services, etc.).

Examples of reference forms are added below.
1. recognition means for recognizing a business object included in an image;
Acquisition means for acquiring business support information for supporting the business of the store clerk regarding the recognized business object;
output means for outputting the acquired business support information;
A processing device having
2. 1. In the processing device according to 1,
The recognition means is a processing device that recognizes a product as the business object.
3. 2. In the processing device according to 2,
The processing device, wherein the business support information includes display locations of the products.
4. In the processing device according to 2 or 3,
The processing device, wherein the business support information includes information indicating an inventory status of the product.
5. In the processing device according to any one of 2 to 4,
The processing device in which the business support information includes the scheduled arrival date of the product.
6. In the processing device according to any one of 2 to 4,
The recognition means is a processing device for recognizing the business object based on product information that associates the identification information of the product with the features of the appearance of current and past packages of the product.
7. In the processing device according to any one of 1 to 6,
The recognition means is a processing device that recognizes a slip as the business object.
8. 7. In the processing device according to 7,
The processing device, wherein the business support information includes information indicating the work to be done for each slip in the collection agent business.
9. 9. In the processing device according to 8,
The processing device, wherein the business support information includes information indicating a barcode written on each slip that needs to be read in the business of the receipt agent.
10. 10. In the processing device according to any one of 1 to 9,
The output means is a processing device that displays the business support information on a display directed to a store clerk.
11. the computer
a recognition step of recognizing business objects included in the image;
an obtaining step of obtaining, from a storage means, business support information for supporting the business of the store clerk regarding the recognized business object;
an output step of outputting the acquired business support information;
The processing method to perform.
12. the computer,
recognition means for recognizing business objects included in the image;
Acquisition means for acquiring, from a storage means, business support information for supporting the business of the store clerk regarding the recognized business object;
output means for outputting the acquired business support information;
A program that acts as

1A processor 2A memory 3A input/output I/F
4A peripheral circuit 5A bus 1 server 2 terminal device 10 processing device 11 recognition unit 12 acquisition unit 13 output unit

Claims (11)

recognition means for recognizing a business object included in an image by image analysis of the image;
an acquisition means for acquiring the business support information corresponding to the recognized business object by using a storage means for storing business support information for supporting the business of the store clerk regarding the business object for each business object;
output means for outputting the acquired business support information;
has
The recognition means is
recognizing a slip as the business object;
A processing device for recognizing the slip using features of the appearance of the slip. The processing apparatus of claim 1,
The recognition means is a processing device that further recognizes a product as the business object. The processing apparatus according to claim 2,
The processing device, wherein the business support information includes display locations of the products. In the processing apparatus according to claim 2 or 3,
The processing device, wherein the business support information includes information indicating an inventory status of the product. In the processing apparatus according to any one of claims 2 to 4,
The processing device in which the business support information includes the scheduled arrival date of the product. In the processing apparatus according to any one of claims 2 to 4,
The recognition means is a processing device for recognizing the business object based on product information in which the identification information of the product is associated with the appearance features of current and past packages of the product. The processing apparatus of claim 1,
The processing device, wherein the business support information includes information indicating the work to be done for each slip in the collection agent business. In the processing apparatus according to claim 7,
The processing device, wherein the business support information includes information indicating a barcode written on each slip that needs to be read in the business of the receipt agent. In the processing apparatus according to any one of claims 1 to 8,
The output means is a processing device that displays the business support information on a display directed to a store clerk. the computer
a recognition step of recognizing a business object included in an image by image analysis of the image;
an acquisition step of acquiring, from the storage means, the business support information corresponding to the recognized business object, using a storage means for storing, for each business object, business support information for supporting the work of a store clerk regarding the business object;
an output step of outputting the acquired business support information;
and run
In the recognition step, a processing method for recognizing a slip as the business object by using features of the appearance of the slip. the computer,
recognition means for recognizing business objects included in an image by image analysis of the image;
Acquisition means for acquiring, from the storage means, the business support information corresponding to the recognized business object, using a storage means for storing business support information for supporting the work of the store clerk regarding the business object for each business object;
output means for outputting the acquired business support information;
function as
The recognition means is
recognizing a slip as the business object;
A program for recognizing the voucher using features of the voucher's appearance.

Images