WO2015039601A1

WO2015039601A1 - Methods, devices, terminal device and systems for pattern recognition

Info

Publication number: WO2015039601A1
Application number: PCT/CN2014/086726
Authority: WO
Inventors: Junbin Kuang
Original assignee: Tencent Technology (Shenzhen) Company Limited
Priority date: 2013-09-22
Filing date: 2014-09-17
Publication date: 2015-03-26
Also published as: CN104143105A

Abstract

Embodiments of the disclosure disclose a method for pattern recognition, as well as relevant device and terminal device, in which the process comprises steps of: scanning the target picture, analyzing the target picture on basis of preset graphic code processing library； if the analysis can get matched graphic code information, determining that the target picture is graphic code and exporting corresponding graphic code information obtained in the analysis； if the analysis fails, photographing the target picture to obtain a photo under identification comprising the target picture, transmitting the obtained photo under identification to a server and receiving returned information corresponding to features of the identified picture after the server identifies picture features of the target picture. The disclosure enables to avoid waste in software and hardware caused by switch in prior art, reduce cost and save time for user.

Description

METHODS, DEVICES, TERMINAL DEVICE AND SYSTEMS FOR PATTERN RECOGNITION

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims priority to Chinese Patent Application No. 201310432743.1, filed on September 22, 2013, which is hereby incorporated by reference in its entirety.

FIELD

The present disclosure relates to the technical field of pattern processing, particularly to a process, device, terminal device and system for pattern recognition.

BACKGROUND

With development of electronic technology and computer technology, for example, telephone, tablet PC and intelligent wearable devices, as well as other terminals with camera function which have been greatly popularized, photographing is rarely limited by time and location, and the requirement of users in photographing has been satisfied.

The development in graphic code technology and pattern recognition technology makes it possible to get information from graphic code and picture by scanning the graphic code and picture with camera, in order to obtain information of the corresponding commodity or add information of contacts according to the graphic code and picture.

However, on condition of prior terminal, the realization of scanning, processing and obtaining corresponding message on the bar code, QR code and pictures such as cover of book needs switch among corresponding bar code scanning mode, QR code scanning mode and picture scanning mode by user. The switch process may take extra terminal software and hardware source, increase energy consumption and waste user’s time due to the search of corresponding functional module.

SUMMARY

The technical problem solved by embodiments of the disclosure is to provide a process, device, terminal device and system for pattern recognition, which can scan and process bar code and picture uniformly and obtain corresponding information.

In one aspect, embodiments of the disclosure provide a method for pattern recognition. In the method, a server obtains a photo including a target picture from a user terminal. The server analyzes the target picture on basis of preset graphic code processing library. When the server gets corresponding graphic code information in the analysis, it confirms that the target picture is graphic code and exports the corresponding graphic code information obtained in the analysis. When the analysis fails, the server identifies the picture features of the target picture and returns information corresponding to features of the identified picture to the user terminal.

In another aspect, embodiments of the disclosure provide one more method for pattern recognition, which includes steps of: scanning a target picture, and analyzing the target picture on basis of preset graphic code processing library； when matched graphic code information is obtained in the analysis, determining the target picture as graphic code and transmitting the matched graphic code information obtained in the analysis； and when the analysis fails, photographing the target picture to obtain a photo under identification comprising the target picture, transmitting the photo under identification to server and receiving the information of content matched with features of the identified picture and returned by the server after identification of picture features of the target picture.

In yet another aspect, embodiments of the disclosure also provide a device for pattern recognition. The device includes a processor and a non-transitory storage medium. The non-transitory storage medium includes modules including: a receiving module, an analyzing module, and a processing module. The receiving module is configured to receive photo under identification obtained in photographing a target picture and comprised target picture. The analyzing module is configured to analyze target picture comprised in photo under identification on basis of preset graphic code processing library. The processing module is configured to determine the target picture as a graphic code and output the matched graphic code information obtained in the analysis on condition that matched graphic code is obtained in analysis； on condition that the analysis fails, identify picture features of the target picture and return content information matched with features of the identified picture to the user terminal.

In yet another aspect, embodiments of the disclosure also provide a device for pattern recognition. The device includes: a scanning module, an outputting module, and a transmitting module. The scanning module is configured to scan target picture and analyze the target picture on basis of preset graphic code processing library. The outputting module is configured to determine the target picture as a graphic code and output the matched graphic code information obtained in the analysis on condition that matched graphic code is obtained in analysis. The transmitting module is configured to photograph the target picture to obtain a photo under identification comprising the target picture, transmit the photo under identification to a server and receive content information matched with identified picture features and returned by the server after identification of picture features for the target picture, on condition that the analysis fails.

In another aspect, embodiments of the disclosure also provide a kind of terminal device that includes a photographic equipment and display. The processor is configured to control the photographic equipment to scan target picture and analyze the target picture on basis of preset graphic code processing library； on condition that matched graphic code information is obtained in the analysis, determine the target picture as graphic code and transmit the matched graphic code information obtained in the analysis； on condition that the analysis fails, photograph the target picture to obtain a photo under identification comprising the target picture, transmit the photo under identification to server and receive the information of content matched with features of the identified picture and returned by the server after identification of picture features of the target picture. The display is configured to display the output of the matched graphic code information obtained in the analysis or display the content information returned by the server and matched with picture features of identified picture.

In yet another aspect, embodiments of the disclosure also provide a pattern recognition system that includes a user terminal and server. The user terminal is configured to photograph a target picture to obtain a photo under identification comprising the target picture and transmit the photo under identification. The server is configured to analyze target picture comprised in the photo under identification on basis of preset graphic code processing library. The server is also configured to determine the target picture as a graphic code and output the matched graphic code information obtained in the analysis on condition that matched graphic code is obtained in analysis； on condition that the analysis fails, identify picture features of the target picture and return content information matched with features of the identified picture to the user terminal.

Embodiments of the disclosure can execute scanning and processing on graphic codes such as bar code and QR code, as well as picture identification and processing, with one scanner, by analyzing graphic code or identifying picture features. There is no need of function switch among identifications of bar code, QR code and cover of book, as well as other target pictures. It avoids waste in source of hardware and software in switch process of prior art, reduces cost and saves time of user.

BRIEF DESCRIPTION OF THE DRAWINGS

Accompanying drawings used in the embodiments and prior art are briefly introduced to make further illustration on the technical scheme of embodiments of the disclosure and prior art. It is apparent that the described accompanying drawings are only specific embodiments of the disclosure. Person of skill in the art may get other accompanying drawings according to the drawings above without any creative labor.

Fig. 1 shows the flow diagram of a pattern recognition method for embodiments of the disclosure.

Fig. 2 shows the flow diagram of another pattern recognition method for embodiments of the disclosure.

Fig. 3 shows the flow diagram of further pattern recognition method for embodiments of the disclosure.

Fig. 4 shows the schematic diagram of a scanning interface for embodiments of the disclosure.

Fig. 5 shows the schematic diagram of a user’s display interface for embodiments of the disclosure.

Fig. 6 shows the structural diagram of a pattern recognition system for embodiments of the disclosure.

Fig. 7 shows the structural diagram of a pattern recognition device for embodiments of the disclosure.

Fig. 8 shows the structural diagram of another pattern recognition device for embodiments of the disclosure.

Fig. 9 shows the structural diagram of further pattern recognition device for embodiments of the disclosure.

Fig. 10 shows the structural diagram of the transmitting module in Fig. 9.

Fig. 11 shows the structural diagram of a kind of terminal device for embodiments of the disclosure.

DETAILED DESCRIPTION OF THE DRAWINGS

Reference throughout this specification to "embodiments, " "an embodiment, " "example embodiment, " or the like in the singular or plural means that one or more particular features, structures, or characteristics described in connection with an embodiment is included in at least embodiments of the present disclosure. Thus, the appearances of the phrases "in embodiments" or "in an embodiment," "in an example embodiment, " or the like in the singular or plural in various places throughout this specification are not necessarily all referring to the same embodiment. Furthermore, the particular features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.

The terminology used in the description of the disclosure herein is for the purpose of describing particular examples only and is not intended to be limiting of the disclosure. As used in the description of the disclosure and the appended claims, the singular forms "a, " "an, " and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. Also, as used in the description herein and throughout the claims that follow, the meaning of “in” includes “in” and “on” unless the context clearly dictates otherwise. It will also be understood that the term "and/or" as used herein refers to and encompasses any and all possible combinations of one or more of the associated listed items. It will be further understood that the terms "may include, " "including, " "comprises, " and/or "comprising, " when used in this specification, specify the presence of stated features, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, operations, elements, components, and/or groups thereof.

As used herein, the term “module” or “unit” may refer to, be part of, or include an Application Specific Integrated Circuit (ASIC) ； an electronic circuit； a combinational logic circuit； a field programmable gate array (FPGA) ； a processor (shared, dedicated, or group) that executes code； other suitable hardware components that provide the described functionality； or a combination of some or all of the above, such as in a system-on-chip. The term module or unit may include memory (shared, dedicated, or group) that stores code executed by the processor.

The exemplary environment may include a server, a client, and a communication network. The server and the client may be coupled through the communication network for information exchange, such as sending/receiving identification information, sending/receiving data files such as splash screen images, etc. Although only one client and one server are shown in the environment, any number of terminals or servers may be included, and other devices may also be included.

The communication network may include any appropriate type of communication network for providing network connections to the server and client or among multiple servers or clients. For example, communication network may include the Internet or other types of computer networks or telecommunication networks, either wired or wireless. In a certain embodiment, the disclosed methods and apparatus may be implemented, for example, in a wireless network that includes at least one client.

In some cases, the client may refer to any appropriate user terminal with certain computing capabilities, such as a personal computer (PC) , a work station computer, a server computer, a hand-held computing device (tablet) , a smart phone or mobile phone, or any other user-side computing device. In various embodiments, the client may include a network access device. The client may be stationary or mobile.

A server, as used herein, may refer to one or more server computers configured to provide certain server functionalities, such as database management and search engines. A server may also include one or more processors to execute computer programs in parallel.

The solutions in the embodiments of the present disclosure are clearly and completely described in combination with the attached drawings in the embodiments of the present disclosure. Obviously, the described embodiments are only a part, but not all, of the embodiments of the present disclosure. On the basis of the embodiments of the present disclosure, all other embodiments acquired by those of ordinary skill in the art under the precondition that no creative efforts have been made shall be covered by the protective scope of the present disclosure.

In order to make the objectives, technical solutions, and advantages of the present disclosure more comprehensible, the present disclosure is further described in detail below with reference to embodiments and the accompanying drawings.

Obviously, embodiments are only described partly below, rather than all of them. Any other embodiment made by person of skill in the art without creative labor is in protection of the disclosure.

Reference is made to Fig. 1, which shows the flow diagram of a pattern recognition method for embodiments of the disclosure. In embodiments of the disclosure, interaction between user terminal and server is applied to realize the pattern recognition process. Specifically, the process comprises the followings.

S101: A server obtains a photo comprising a target picture from a user terminal. The user terminal includes a camera and photographs a target picture to obtain the photo under identification including the target picture and transmits the photo under identification to the server.

The user terminal may be a tablet PC, telephone, E-Reader, remote control, PC, laptop, mobile unit, network television, wearable device and other intellectual terminals with camera. Target pictures, such as bar code, QR code, as well as some book covers and commodity pictures, can be photographed by calling a photographing module, to obtain a photo under identification comprising the target picture. And then, the photo under identification is transmitted to a corresponding server through communication network or internet.

Specifically, the step of S101 may comprise: a user terminal performs a motion estimation of terminal； and if the result of motion estimation shows that the user terminal is in the state of motionless, triggering photographing the target picture and obtaining a photo under identification comprising the target picture； and transmitting the photo under identification to a server.

S102: The server analyzes the target picture on basis of preset graphic code processing library.

S103: If the server can get corresponding graphic code information in the analysis, it determines that the target picture is graphic code and exports corresponding graphic code information obtained in the analysis； if the analysis fails, the server identifies the picture features of the target picture and returns information corresponding to features of the identified picture to the user terminal.

The server may be application server corresponding to application of the process described in embodiments of the disclosure, or a preset function server, for example a server for identification of picture, book or commodity. Specifically, the server may be set by pre-allocating IP address of the server, pot number and the like. So that, user can transmit the photo under identification to the server conveniently.

When the server carries out analysis on the scanned target picture on basis of preset graphic code processing library, if the analysis gets corresponding graphic code information, it indicates that the target picture is bar code or QR code, and the server returns the graphic code information matched with the graphic code, namely the target picture, to the user terminal.

When the server carries out analysis on the scanned target picture on basis of preset graphic code processing library, if the analysis gets nothing, it indicates that the analysis fails and confirms that the target picture is a picture such as book cover, commodity picture, or the like. Then, picture identification mode is applied to identify picture features and carry out searching and processing on basis of the picture features. Thus, content information matched with the identified picture features, namely information of book, commodity or the like corresponding to the target picture, is obtained. For example, if it is confirmed that title of the book comprises “Chinese Dictionary” in a target picture of a book cover, the search process will be started on basis of the “Chinese Dictionary” to get various information related to the Chinese Dictionary.

After the server gets information related to the target picture, it returns content information matched with the identified picture features to the user terminal. As well, the user terminal organizes and displays the information related to the target picture and returned by the server through a display interface comprising text, picture and link address.

Reference is made to Fig. 2, which shows the flow diagram of another pattern recognition method for embodiments of the disclosure. The process described in the preferred embodiments of the disclosure may be applied in a tablet PC, telephone, E-Reader, remote control, PC, laptop, mobile unit, network television, wearable device and other intellectual terminals with camera. Specifically, the process comprises steps as below.

S201: Scanning target picture and analyzing the target picture on basis of preset graphic code processing library.

After the user terminal finds that the terminal user has unlocked the pattern scanning function, it calls the system camera to scan the target picture locating in current camera lens position. The target picture may be graphic code comprising information data, such as bar code or QR code, or some book cover and commodity picture and the like.

The preset graphic code processing library comprises preset graphic code processing library configured to process QR code and/or bar code. With the graphic code processing library, the terminal can analyze the QR code or bar code, and the information data carried in target picture, which is QR code or bar code, can be obtained.

S202: On condition that matched graphic code information is obtained in the analysis, determining the target picture as graphic code and transmitting the matched graphic code information obtained in the analysis.

When the Step 201 carries out analysis on the scanned target picture on basis of preset graphic code processing library as descried in S201, if the analysis of user terminal gets corresponding graphic code information, it indicates that the target picture is bar code or QR code, and the matched graphic code information obtained in the analysis will be output as result of the scanning and processing.

Particularly, the graphic code information of the target picture, output in the step of S202, can be organized and displayed through a display interface comprising text, picture and link address.

S203: On condition that the analysis fails, photographing the target picture to obtain a photo under identification comprising the target picture, transmitting the photo under identification to server and receiving the information of content matched with features of the identified picture and returned by the server after identification of picture features of the target picture.

When the server carries out analysis on the scanned target picture on basis of preset graphic code processing library, if the analysis gets nothing, it indicates that the analysis fails and confirms that the target picture is a picture such as book cover, commodity picture, or the like. Then, a clear target picture is obtained thorough photographing and transmitted to a server which is able to identify picture, book and commodity.

The server uses current picture identification process to identify the picture features, and searches and processes on basis of the picture features. Thus, content information matched with the identified picture features, namely information of book, commodity or the like corresponding to the target picture, is obtained. For example, if it is confirmed that title of the book comprises “Chinese Dictionary” in a target picture of a book cover, the search process is started on basis of the “Chinese Dictionary” to get various information related to the “Chinese Dictionary” . After the server gets information related to the target picture, it returns content information matched with the identified picture features to the user terminal. As well, the user terminal organizes and displays the information related to the target picture and returned by the server through a display interface comprising text, picture and link address.

Embodiments of the disclosure can execute such as bar code and QR code, as well as picture identification and processing, with one scanner, by analyzing graphic code or identifying picture features. There is no need of function switch among identifications of bar code, QR code and cover of book, as well as other target pictures. It avoids waste in source of hardware and software in switch process of prior art, reduces cost and saves time of user.

Reference is made again to Fig. 3, which shows the flow diagram of further pattern recognition method for embodiments of the disclosure. The process described in the preferred embodiments of the disclosure may be applied in a tablet PC, telephone, E-Reader, remote control, PC, laptop, mobile unit, network television, wearable device and other intellectual terminals with camera. Specifically, the process comprises steps of followings.

S301: When scanning is detected, triggering start of the terminal scanning module and calling the scanning interface for user, wherein there is one or more buttons, which may be pattern scanning button, translation scanning button and street view scanning button in the scanning interface.

Specifically, the scanning interface may be a human-computer interface as shown in Fig. 4. On the interface, user can make a selection among scanning graphic code such as the bar code/QR code, scanning picture such as picture of book cover/commodity under identification, scanning English/Chinese words in picture under translation, and scanning street view for determining location of terminal.

S302: On condition that an operation event of selection on pattern scanning button is detected, executing the operation of scanning target picture and analyzing the target picture on basis of preset graphic code processing library, wherein the preset graphic code processing library comprises preset graphic code processing library configured to process QR code and/or bar code.

On condition that the user terminal detects an operation event that pattern scanning button is selected, it will call a system camera to scan the target picture on the position under the camera lens, in which the target picture may be bar code, QR code, or other graphic codes with information data, or may be pictures, such as book cover or commodity picture. With the graphic code processing library, the terminal can analyze the QR code or bar code, so that information data carried in the target picture comprising the QR code or bar code will be obtained.

S303: On condition that matched graphic code information is obtained in the analysis, identifying the target picture as graphic code and transmitting the matched graphic code information obtained in the analysis.

When user terminal carries out analysis on the scanned target picture on basis of preset graphic code processing library, if the analysis of user terminal gets corresponding graphic code information, it indicates that the target picture is bar code or QR code, and the matched graphic code information obtained in the analysis will be output as result of the scanning and processing.

S304: On condition that the analysis fails, executing motion estimation onto a terminal； if the result of motion estimation shows the terminal is in state of motionless, triggering photographing the target picture, in order to obtain a photo under identification comprising the target picture.

S305: Transmitting the photo under identification to a server, so that the server can complete the identification on picture features of the target picture.

The user terminal gets clear target picture by photographing and the motion estimation is configured to judge whether the terminal has changed from motion state to motionless state. The picture should be photographed in the state of motionless to avoid fuzziness caused by moving or shaking, and to obtain high-quality photo under identification of the target picture.

In the Step S305, the server may include an application server corresponding to application of the process described in embodiments of the disclosure, or a preset function server, for example a server for identification of picture, book or commodity. Specifically, the server may be set by pre-allocating IP address of the server, pot number and the like. So that, a user device may transmit the photo under identification to the server conveniently.

The server uses current picture identification process to identify the picture features, and searches and processes on basis of the picture features. Thus, content information matched with the identified picture features, namely information of book, commodity or the like corresponding to the target picture, is obtained.

S306: Receiving the information of content matched with features of the identified picture and returned by the server after identification of picture features of the target picture. The S304 and S306 are corresponding to the S203 in the previous embodiment.

S307: On condition that an operation event of selection on translation scanning button is detected, executing the operation of photographing a target picture and transmitting the photographed pattern to a server.

In the Step S307, the server may be one used for translation. When user terminal detects that the user has selected the translation scanning button, there is no need to test whether the picture could be analyzed on basis of graphic code processing library. Photograph can be executed directly to obtain corresponding picture. As well, motion estimation may be applied in photographing the picture when the terminal is in state of motionless. The server can identify content in picture and execute translation. For example, it identifies each Chinese character on basis of character features in the picture and completes the translation from Chinese to English. Specifically, the server can be arranged by setting the IP address.

S308: On condition that an operation event of selection on street view scanning button is detected, executing the operation of photographing a target picture and transmitting the photographed street view picture to a server.

In the Step S308, the server may be one configured to locate the position according to a picture. When user terminal detects that the user has selected the street view scanning button, there is no need to test whether the picture can be analyzed on basis of graphic code processing library. Photographing can be executed directly to obtain corresponding picture. As well, motion estimation may be applied in photographing the picture when the terminal is in state of motionless. The server can identify the content of street view in a picture and judge the geographic position of a building shown in the picture or marked with some words, by which the server determines the location of terminal. For example, it is judged that the target picture is photographed in a gymnasium according to the features of building in a picture, by which it can be determined that the current position is in the gymnasium. Specifically, the server can be arranged by setting the IP address.

S309: Generating a user interface for user, wherein the user interface is configured to display the output of the matched graphic code information obtained in analysis, or configured to display the content information returned by the server and matched with identified picture features.

Specifically, the generated user display interface may be one as shown in Fig. 5, through which the user terminal organizes graphic code information or information returned by a server for picture analysis, for translation or for determining position, and shows to the user. It should be aware that in the Steps S305, S307 and S308, the involved server may be one with several functions.

Embodiments of the disclosure can execute scanning and processing on graphic codes such as bar code and QR code, as well as picture identification and processing, with one scanner, by analyzing graphic code or identifying picture features. There is no need of function switch among identifications of bar code, QR code and cover of book, as well as other target pictures. It avoids waste in source of hardware and software in switch process of prior art, reduces cost and saves time of user. Meanwhile, the functions of translation and street scanning can be executed in the scanner, which further avoids the waste in software and hardware, and reduces the cost. A human-computer interface is configured to prompt user to choose a right function to get corresponding information in target picture, which helps user complete the operation visually and conveniently.

The pattern recognition system, device and terminal device for embodiments of the disclosure are described in detail as below.

With reference to the Fig. 6, which shows the structural diagram of a pattern recognition system for embodiments of the disclosure, the pattern recognition system, used in embodiments of the disclosure includes a user terminal 100 and a server 200.

The user terminal 100 is configured to photograph a target picture to obtain a photo under identification comprising the target picture and transmit the photo under identification to the server 200；

The server 200 is configured to analyze the target picture comprised in the photo under identification on basis of preset graphic code processing library； and

the server 200 is also configured to determine the target picture as a graphic code and output the matched graphic code information obtained in the analysis on condition that matched graphic code is obtained in analysis； on condition that the analysis fails, identify picture features of the target picture and return content information matched with features of the identified picture to the user terminal 200.

The user terminal 100 may be a tablet PC, telephone, E-Reader, remote control, PC, laptop, mobile unit, network television, wearable device and other intellectual terminals with camera. Target pictures, such as bar code, QR code, as well as some book covers and commodity pictures, can be photographed by calling a photographing module, to obtain a photo under identification comprising the target picture. And then, the photo under identification will be transmitted to a corresponding server through communication network or internet.

Specifically, the user terminal 100 is configured to carry out motion estimation of terminal, and if the result of motion estimation shows that the user terminal is in the state of motionless, it triggers off photographing the target picture, obtains a photo under identification comprising the target picture and transmits the photo under identification to a server.

The server 200 may be an application server corresponding to application of the process described in embodiments of the disclosure, or a preset function server, for example a server for identification of picture, book or commodity. Specifically, the server may be set by pre-allocating IP address of the server, pot number and the like. Thus, a user may transmit the photo under identification to the server 200 conveniently.

When the server 200 carries out analysis on the scanned target picture on basis of preset graphic code processing library, if the analysis gets corresponding graphic code information, it indicates that the target picture is bar code or QR code, and the server returns the graphic code information matched with the graphic code, namely the target picture, to the user terminal.

When the server 200 carries out analysis on the scanned target picture on basis of preset graphic code processing library, if the analysis gets nothing, it indicates that the analysis fails and confirms that the target picture is a picture such as book cover, commodity picture, or the like. Then, picture identification mode is applied to identify picture features and carry out searching and processing on basis of the picture features. Thus, content information matched with the identified picture features, namely information of book, commodity or the like corresponding to the target picture, will be obtained. For example, if it is confirmed that title of the book comprises Chinese Dictionary in a target picture of a book cover, the search process will be started on basis of the “Chinese Dictionary” to get various information related to the “Chinese Dictionary” .

After the server 200 gets information related to the target picture, it returns content information matched with the identified picture features to the user terminal. As well, the user terminal organizes and displays the information related to the target picture and returned by the server 200 through a display interface comprising text, picture and link address.

Reference is made to Fig. 7, which shows the structural diagram of a pattern recognition device 700 for embodiments of the disclosure. In embodiments of the disclosure, the device 700 may be installed in various application servers. Specifically, the device 700 may include a hardware processor 710 and a non-transitory storage 720 configured to store the following modules:

a receiving module 01, configured to receive photo under identification obtained in photographing a target picture and comprised target picture；

an analyzing module 02, configured to analyze target picture comprised in photo under identification on basis of preset graphic code processing library； and

a processing module 03, configured to identify the target picture as a graphic code and output the matched graphic code information obtained in the analysis on condition that matched graphic code is obtained in analysis； on condition that the analysis fails, identify picture features of the target picture and return content information matched with features of the identified picture to the user terminal.

After the receiving module 01 gets the photo under identification, the analyzing module 02 carries out analysis on the scanned target picture on basis of preset graphic code processing library and if the processing module 03 gets matched graphic code information during the analysis, it indicates that the target picture is bar code or QR code, and the server returns the graphic code information matched with the graphic code, namely the target picture, to the user terminal.

When the analyzing module 02 carries out analysis on the scanned target picture on basis of preset graphic code processing library, if the analysis gets nothing, it indicates that the analysis fails, and then the processing module 03 confirms that the target picture is a picture such as book cover, commodity picture, or the like. Then, the processing module 03 gets into picture identification mode to identify picture features and carries out searching and processing on basis of the picture features. Thus, content information matched with the identified picture features, namely information of book, commodity or the like corresponding to the target picture, is obtained. For example, if it is confirmed that title of the book comprises “Chinese Dictionary” in a target picture of a book cover, the search process will be started on basis of the Chinese Dictionary to get various information related to the “Chinese Dictionary” .

After the processing module 03 gets information related to the target picture, it returns content information matched with the identified picture features to the user terminal. As well, the user terminal organizes and displays the information related to the target picture and returned by the server through a display interface comprising text, picture and link address.

Reference is made to Fig. 8, which is the structural diagram of another pattern recognition device 800 for embodiments of the disclosure. The device 800 introduced in embodiments of the disclosure may be installed in a tablet PC, telephone, E-Reader, remote control, PC, laptop, mobile unit, network television, wearable device and other intellectual terminals with camera. For example, the device 800 includes a hardware processor 810 and a non-transitory storage medium 820 accessible to the hardware processor 810. The non-transitory storage medium 820 is configured to store modules including:

a scanning module 1, configured to scan target picture and analyze the target picture on basis of preset graphic code processing library；

an outputting module 2, configured to identify the target picture as a graphic code and output the matched graphic code information matched obtained in the analysis on condition that matched graphic code is obtained in analysis； and

a transmitting module 3, configured to photograph the target picture to obtain a photo under identification comprising the target picture, transmit the photo under identification to a server and receive content information matched with identified picture features and returned by the server after it identifies picture features of the target picture, on condition that the analysis fails.

After the user terminal detects the pattern scanning function unlocked, the scanning module 1 calls the system camera to scan the target picture locating in current camera lens position. The target picture may be graphic code comprising information data, such as bar code or QR code, or some book cover and commodity picture and the like.

The preset graphic code processing library comprises preset graphic code processing library configured to process QR code and/or bar code. With the graphic code processing library, the scanning module 1 analyzes the QR code or bar code, and the information data carried in target picture, which is QR code or bar code, is obtained.

When the scanning module 1 carries out analysis on the scanned target picture on basis of preset graphic code processing library, if the analysis of user terminal gets corresponding graphic code information, it indicates that the target picture is bar code or QR code, and the matched graphic code information obtained in the analysis will be output as result of the scanning and processing.

Specifically, a display interface comprising text, picture and link address can be configured to organize and display the information related to the target picture and output by the outputting module 2.

When the scanning module 1 carries out analysis on the scanned target picture on basis of preset graphic code processing library, if the analysis gets nothing, it indicates that the analysis fails and confirms that the target picture is a picture such as book cover, commodity picture, or the like. Then, a clear target picture can be obtained by transmission module 3 thorough photographing and transmitted to a server which is able to identify picture, book and commodity. The server can be set beforehand by IP address allocation.

The server takes advantage of the current pattern recognition approach to identify picture features and executes searching and processing on basis of the picture features, in order to get the specific information of book or commodity corresponding to the target picture. For example, if it is confirmed that title of the book comprises “Chinese Dictionary” in a target picture of a book cover, the search process will be started on basis of the “Chinese Dictionary” to get various information related to the “Chinese Dictionary” . After the server gets information related to the target picture, it returns content information matched with the identified picture features to the user terminal. As well, the user terminal organizes and displays the information related to the target picture and returned by the server through a display interface comprising text, picture and link address.

With reference to Fig. 9, which shows the structural diagram of a pattern recognition device 900 for embodiments of the disclosure, the device 900 introduced in embodiments of the disclosure may be installed in a tablet PC, telephone, E-Reader, remote control, PC, laptop, mobile unit, network television, wearable device and other intellectual terminals with camera. Specifically, the device 900 includes a processor 910 and a non-transitory storage medium 920 configured to store modules including the scanning module 1, the outputting module 2 and the transmitting module 3. Particularly, in embodiments of the disclosure, the device 900 may further include:

an interface calling module 4, configured to trigger start of the terminal scanning module and call the scanning interface for user when scanning is detected, in which there is one or more buttons, which may be pattern scanning button, translation scanning button and street view scanning button in the scanning interface； and

a processing module 5, configured to inform the scanning module on condition that an operation event of selection on pattern scanning button is detected； and on condition that an operation event of selection on translation scanning button is detected, to photograph target picture and transmit the photographed pattern to a server； and on condition that an operation event of selection on street view scanning button is detected, to photograph target picture and transmit the photographed street view picture to a server.

The scanning interface called by the interface calling interface 4 may be a human-computer interface as shown in Fig. 4. On the interface, user can make a selection among scanning graphic code such as the bar code/QR code, scanning picture such as picture of book cover/commodity under identification, scanning English/Chinese words in picture under translation, and scanning street view for determining location of terminal.

The processing module 5 informs the scanning module 1 to execute scanning on the target picture in case of an operation event of selection on pattern scanning button, so that the outputting module 2 or transmitting module 3 can continue corresponding functions.

When the user selects the translation scanning button, there is no need to test whether the picture can be analyzed on basis of graphic code processing library. Photographing is executed by the processing module 5 directly to obtain corresponding picture. As well, motion estimation is applied in photographing the picture when the terminal is in state of motionless. The server identifies content in picture and translates the identified content from a first language to a second language. For example, the server may identify each Chinese character on basis of character features in the picture and complete the translation from Chinese to English. Specifically, the server can be arranged by setting the IP address.

When the user selects the street view scanning button, there is no need to test whether the picture could be analyzed on basis of graphic code processing library. Photographing is executed by the processing module 5 directly to obtain corresponding picture. As well, motion estimation is applied in photographing the picture when the terminal is in state of motionless. The server identifies the content of street view in a picture and judge the geographic position of a building shown in the picture or marked with some words, by which the server determines the location of terminal. For example, it is judged that the target picture is photographed in a gymnasium according to the features of building in a picture, by which it can be determined that the current position is in the gymnasium. Specifically, the server can be arranged by setting the IP address.

Reference is made further to Fig. 10 and the transmitting module 3 may specifically include:

a triggering unit 31, configured to execute motion estimation onto a terminal on condition that the analysis fails and, if the result of motion estimation shows the terminal is in state of motionless, trigger off photographing the target picture, in order to obtain a photo under identification comprising the target picture； and

a transmitting unit 32, configured to transmit the photo under identification to a server in order to get the server completing picture feature identification of the target picture.

When the scanning module 1 carries out analysis on the scanned target picture based on preset graphic code processing library, if the analysis gets nothing, it indicates that the analysis fails and confirms that the target picture is a picture such as book cover, commodity picture, or the like. The user terminal gets clear target picture by triggering photographing with unit 31 and the motion estimation is configured to judge whether the terminal is changed from motion state to motionless state. The picture should be photographed in the state of motionless to avoid fuzziness caused by moving or shaking and to obtain high-quality photo under identification of the target picture.

The transmitting unit 32 transmits the photo under identification to a server, which is able to identify picture, book and commodity. The server can be set beforehand by IP address allocation.

The server takes advantage of the current pattern recognition approach to identify picture features and executes searching and processing on basis of the picture features in order to get the specific information of book or commodity corresponding to the target picture.

Further, the device described in embodiments of the disclosure may include generating module 6, configured to generate a user interface for user, wherein the user interface is configured to display the output of the matched graphic code information obtained in analysis, or to display the content information returned by the server and matched with identified picture features.

The generated user display interface may be one as shown in Fig. 5, through which the user terminal organizes graphic code information or information returned by a server for picture analysis, for translation or for determining position, and shows to the user.

With further reference to Fig. 11, which shows the structural diagram of an example terminal device for embodiments of the disclosure,

Fig. 11 is a block diagram of a partial device related to a terminal device such as a mobile phone. For example, the terminal device includes a radio frequency (RF) circuit 510, a memory 520, an input unit 530, a display unit 540, a sensor 550, an audio circuit 560, a wireless fidelity (WiFi) module 570, a processor 580, and a power 590, etc. . It’s understood for persons skilled in the art that, the structure of a terminal device illustrated in Fig. 11 is not limited, some components can be added or omitted, or some combinations or arrangement can be included.

Following is a detailed description of the structure of the terminal device by combining with Fig. 11.

The RF circuit 510 is configured to receive and sending signals during calling or process of receiving and sending message. Specially, the RF circuit 510 will receive downlink information from the base station and send it to the processor 580； or send uplink data to the base station. Generally, the RF circuit 510 includes, but is not limited to, an antenna, at least one amplifier, a transceiver, a coupler, a low noise amplifier (LNA) , a diplexer, and the like. In addition, the RF circuit 40 can communicate with network or other devices by wireless communication. Such wireless communication can use any one communication standard or protocol, which includes, but is not limited to, Global System of Mobile communication (GSM) , (General Packet Radio Service, GPRS) , (Code Division Multiple Access, CDMA) , (Wideband Code Division Multiple Access, WCDMA) , (Long Term Evolution, LTE) , email, or (Short Messaging Service, SMS) .

The memory 520 is configured to store software program and module which will be run by the processor 580, so as to perform multiple functional applications of the terminal device and data processing. The memory 530 mainly includes storing program area and storing data area. For example, the storing program area can store the operating system, at least one application program with required function (such as sound playing function, image playing function, etc. ) . The storing data area can store data established by terminal device according to actual using demand (such as audio data, phonebook, etc. ) Furthermore, the memory 520 can be high-speed random access memory, or nonvolatile memory, such as disk storage, flash memory device, or other volatile solid-state memory devices.

The input unit 530 is configured to receive the entered number or character information, and the entered key signal related to user setting and function control of the terminal device 500. For example, the input unit 530 includes at least one of: a touch panel 531, other input devices 532, and a camera 533.

The touch panel 531 may include a touch screen, which can collect user’s touch operations thereon or nearby (for example the operations generated by fingers of user or stylus pen, and the like, touching on the touch panel 531 or touching near the touch panel 531) , and drive the corresponding connection device according to the preset program. Optionally, the touch panel 531 includes two portions including a touch detection device and a touch controller. Specifically, the touch detection device is configured to detect touch position of the user and detecting signals accordingly, and then sending the signals to the touch controller. Subsequently, the touch controller receives touch information from the touch detection device, and converts it to contact coordinates which are to be sent to the processor 580, and then receives command sent by the processor 580 to perform. In addition, the touch panel 531 can be implemented is forms of resistive type, capacitive type, infrared type and surface acoustic wave type. Besides the touch panel 531, the input unit 530 can include, but is not limited to other input devices 532, such as one or more selected from physical keyboard, function keys (such as volume control keys, switch key-press, etc. ) , a trackball, a mouse, and an operating lever, etc. . The camera 533 may be an optical device configured to record images in a digital format.

The display unit 540 is configured to display information entered by the user or information supplied to the user, and menus of the terminal device. For example, the display unit 540 includes a display panel 541, such as a Liquid Crystal Display (LCD) , or an Organic Light-Emitting Diode (OLED) . Furthermore, the display panel 541 can be covered by the touch panel 531, after touch operations are detected on or near the touch panel 531, they will be sent to the processor 580 to determine the type of the touching event. Subsequently, the processor 580 supplies the corresponding visual output to the display panel 541 according to the type of the touching event. As shown in Fig. 11, the touch panel 531 and the display panel 541 are two individual components to implement input and output of the terminal device, but they can be integrated together to implement the input and output in some embodiments.

Furthermore, the terminal device 500 includes at least one sensor 550, such as light sensors, motion sensors, or other sensors. Specifically, the light sensors includes ambient light sensors for adjusting brightness of the display panel 541 according to the ambient light, and proximity sensors for turning off the display panel 541 and/or maintaining backlight when the terminal device is moved to the ear side. Accelerometer sensor as one of the motion sensors can detect the magnitude of accelerations in every direction (Triaxial, generally) , and detect the magnitude and direction of gravity in an immobile status, which is applicable to applications of identifying attitudes of the mobile (such as switching between horizontal and vertical screens, related games, magnetometer attitude calibration, etc. ) , vibration recognition related functions (such as pedometer, percussion, etc. ) . And the terminal device 500 also can configure other sensors (such as gyroscopes, barometers, hygrometers, thermometers, infrared sensors, etc. ) whose detailed descriptions are omitted here.

The audio circuit 560, the speaker 561 and the microphone 562 supply an audio interface between the user and the terminal device. Specifically, the audio data is received and converted to electrical signals by audio circuit 560, and then transmitted to the speaker 561, which are converted to sound signal to output. On the other hand, the sound signal collected by the speaker is then converted to electrical signals which will be received and converted to audio data. Subsequently, the audio data are output to the processor 580 to process, and then sent to another terminal device via the RF circuit 510, or sent to the memory 520 to process further.

WiFi pertains to short-range wireless transmission technology providing a wireless broadband Internet, by which the terminal device can help the user to receive and send email, browse web, and access streaming media, etc. . Although the WiFi module 570 is illustrated in Fig. 11, it should be understood that, WiFi module 570 is not a necessary for the terminal device, which can be omitted according the actual demand without changing the essence of the present disclosure.

The processor 580 is a control center of the terminal device, which connects with every part of the terminal device by various interfaces or circuits, and performs various functions and processes data by running or performing software program/module stored in the memory 520 or calling data stored in the memory 520, so as to monitor the terminal device. Optionally, the processor 580 may include one or more processing units. Preferably, the processor 580 can integrate with application processors and modem processors, for example, the application processors include processing operating system, user interface and applications, etc. ； the modern processors are used for performing wireless communication. It can be understood that, it’s an option to integrate the modern processors to the processor 580.

The processor 580 is configured to control a photographic equipment such as the camera 533 to scan target picture and analyze the target picture on basis of preset graphic code processing library； on condition that matched graphic code information is obtained in the analysis, determine the target picture as graphic code and transmit the matched graphic code information obtained in the analysis； on condition that the analysis fails, photograph the target picture to obtain a photo under identification comprising the target picture, transmit the photo under identification to server and receive the information of content matched with features of the identified picture and returned by the server after identification of picture features of the target picture.

The processor 580 is configured to control the display unit 540. The display 540 is configured to display the output of the matched graphic code information obtained in the analysis or display the content information returned by the server and matched with picture features of identified picture.

Furthermore, the terminal device 500 may include a power supply (such as battery) supplying power for each component, preferably, the power supply can connect with the processor 580 by power management system, so as to manage charging, discharging and power consuming.

In addition, the terminal device 500 may include a Bluetooth module and other hardware circuits, etc. , which are not illustrated.

Specifically, the processor 580 may implement the methods in Fig. 2 to Fig. 10.

The whole or part of process in embodiments above may be implemented through relevant hardware under instruction of a computer program, in which the program may be stored in a computer-readable memory medium. When the program is executed, the methods disclosed above may be performed by the relevant hardware, so that scanning and processing for graphic codes such as bar code/QR code, as well as scanning for book cover and other pictures and information determining, can be completed on basis of once scanning. The memory medium above may include a diskette, an optical disk, Read-Only Memory (ROM) , Random Access Memory (RAM) , or the like.

All disclosures above are the preferred embodiments of the disclosure, which is not intending to limit range of the disclosure. Therefore, any equivalent change according to the claims of the disclosure is in range of the disclosure.

Claims

A method for pattern recognition, comprising:

obtaining, by a server, a photo comprising a target picture from a user terminal；

analyzing, by the server, the target picture based on a preset graphic code processing library；

when the server gets corresponding graphic code information in the analysis, determining that the target picture is a graphic code and exporting corresponding graphic code information obtained in the analysis； and

when the analysis fails, identifying picture features of the target picture and returning information corresponding to features of the identified picture to the user terminal.
The method of claim 1, further comprising:

performing a motion estimation of the user terminal； and

if the motion estimation shows that the user terminal is in the state of motionless, triggering photographing the target picture and obtaining the photo comprising the target picture.
A method for pattern recognition, comprising:

scanning, by a terminal device comprising a camera, a target picture, and analyzing the target picture based on a preset graphic code processing library；

when the analysis obtains matched graphic code information, determining the target picture as a graphic code and outputting the graphic code information obtained in the analysis； and

when the analysis fails, photographing the target picture to obtain a photo under identification comprising the target picture, transmitting the photo under identification to a server and receiving the information of content matched with features of the identified picture.
The method of claim 3, further comprising:

performing motion estimation in the terminal device； if the motion estimation shows the terminal is in state of motionless, triggering photographing the target picture, in order to obtain a photo under identification comprising the target picture； and

transmitting the photo under identification to a server, in order to get the server completing picture feature identification for the target picture.
The method of claim 3, further comprising:

when scanning is detected, triggering the terminal to start scanning module and display the scanning interface, the scanning interface comprises at least one of the following: a pattern scanning button, a translation scanning button, and a street view scanning button；

when an operation event of selection on pattern scanning button is detected, executing the operation of scanning target picture and analyzing the target picture on basis of preset graphic code processing library；

when an operation event of selection on translation scanning button is detected, executing the operation of photographing a target picture and transmitting the photographed pattern to a server； and

when an operation event of selection on street view scanning button is detected, executing the operation of photographing a target picture and transmitting the photographed street view picture to a server.
The method of any one of claims 3-5, further comprising:

generating a user interface for user, wherein the user interface is configured to display the output of the matched graphic code information obtained in analysis, or configured to display the content information returned by the server and matched with identified picture features.
The method of claim 3, wherein the preset graphic code processing library comprises preset graphic code processing library configured to process QR code and/or bar code.
A device for pattern recognition, comprising:

a receiving module, configured to receive photo under identification obtained in photographing a target picture and comprising target picture；

an analyzing module, configured to analyze target picture comprised in photo under identification on basis of preset graphic code processing library； and

an processing module, configured to determine the target picture as a graphic code and output the matched graphic code information obtained in the analysis on condition that matched graphic code is obtained in analysis； on condition that the analysis fails, to identify picture features of the target picture and return content information matched with features of the identified picture to the user terminal.
A device for pattern recognition, comprising:

a scanning module, configured to scan target picture and analyze the target picture on basis of preset graphic code processing library；

an outputting module, configured to determine the target picture as a graphic code and output the corresponding graphic code information obtained in the analysis on condition that matched graphic code is obtained in analysis； and

a transmitting module, configured to photograph the target picture to obtain a photo under identification comprising the target picture, transmit the photo under identification to a server and receive content information matched with identified picture features and returned by the server after identifying picture features of the target picture, on condition that the analysis fails.
The device of claim 9, wherein the transmitting module comprises:

a triggering unit, configured to perform a motion estimation onto a terminal on condition that the analysis fails； if the motion estimation shows the terminal is in state of motionless, to trigger off photographing the target picture, in order to obtain a photo under identification comprising the target picture； and

a transmitting unit, configured to transmit the photo under identification to a server, in order to get the server completing picture feature identification of the target picture.
The device of claim 9, further comprising:

an interface calling module, configured to trigger start of the terminal scanning module and call the scanning interface for user when scanning is detected, wherein there is one or more buttons, which may be pattern scanning button, translation scanning button and street view scanning button in the scanning interface； and

an processing module, configured to inform the scanning module on condition that an operation event of selection on pattern scanning button is detected； and on condition that an operation event of selection on translation scanning button is detected, to photograph a target picture and transmit the photographed pattern to a server； and on condition that an operation event of selection on street view scanning button is detected, to photograph a target picture and transmit the photographed street view picture to a server.
The device of any one of claims 9-11, further comprising:

a generating module, configured to generate a user interface for user, wherein the user interface is configured to display the output of matched graphic code information obtained in analysis, or configured to display the content information returned by the server and matched with identified picture features.
The device of claim 9, wherein the preset graphic code processing library comprises the preset graphic code processing library configured to process QR code and/or bar code.
A terminal device, comprising:

a photographic equipment configured to take a picture；

a processor in communication with the photographic equipment； and

a display configured to display at least a portion of the picture, wherein:

the processor is configured to control the photographic equipment to scan target picture and analyze the target picture on basis of preset graphic code processing library； on condition that matched graphic code information is obtained in the analysis, determine the target picture as graphic code and transmit the matched graphic code information obtained in the analysis； on condition that the analysis fails, photograph the target picture to obtain a photo under identification comprising the target picture, transmit the photo under identification to server and receive the information of content matched with features of the identified picture and returned by the server after identification of picture features of the target picture, and the display is configured to display the output of the matched graphic code information obtained in the analysis or display the content information returned by the server and matched with picture features of identified picture.