CN109740597A

CN109740597A - The recognition methods of menu information and device

Info

Publication number: CN109740597A
Application number: CN201811639502.3A
Authority: CN
Inventors: 刘泉; 吴洋; 杨宇; 陈晨; 魏世康; 张依蕾; 许晓吟; 田正中; 兰杰; 朱兴
Original assignee: Koubei Shanghai Information Technology Co Ltd
Current assignee: Koubei Shanghai Information Technology Co Ltd
Priority date: 2018-12-29
Filing date: 2018-12-29
Publication date: 2019-05-10

Abstract

The invention discloses a kind of recognition methods of menu information and devices, wherein method includes: acquisition vegetable text information corresponding with sample vegetable；Presentation style of the vegetable text information in menu picture is simulated, generates simulation menu picture corresponding with vegetable text information according to analog result；Simulation menu picture is determined as menu sample, determines sample markup information corresponding with menu sample；According to menu sample and sample markup information corresponding with menu sample training menu identification model, the recognition result of menu picture to be identified is obtained by menu identification model.In the present invention program, menu identification model is obtained according to simulation menu picture training, wherein, the required sample markup information of training can accurately be determined by vegetable text information, the problem of annotation results inaccuracy can be avoided the occurrence of, the accuracy for being conducive to improve the menu identification model that training obtains, further increases the accuracy of recognition result.

Description

The recognition methods of menu information and device

Technical field

The present invention relates to field of computer technology, and in particular to a kind of recognition methods of menu information and device.

Background technique

Optical character identification (Optical Character Recognition, abbreviation OCR) can be to the text in picture Information extracts, for example Ali's cloud general purpose O CR service can complete the extraction to the text information on picture.Extracting text After word information, artificial intelligence general technology such as tree-model (decision tree, random forest, GBDT, Xgboost etc.), mind can be passed through Analysis identification is carried out to text information through network model (CNN, DNN etc.) etc., obtains the content of text of demand.It is above-mentioned to be needed The process for the content of text asked can realize that the comprehensive text information of the algorithm model extracts function by a set of tailor-made algorithm model It can and analyze identification function.

At the same time, in recent years, as prior restaurant industry is gradually to the development of intelligent direction, by traditional papery dish Menu text in list, which is synchronized in various terminals or server, has great practical value.Therefore, tailor-made algorithm mould is utilized Type, which carries out identification to the text in menu picture, can be considered a kind of effective mode.

In the scheme that the existing menu text in menu picture is identified, algorithm model is usually to pass through such as lower section What formula obtained: acquisition menu picture sample set, and to the various menu picture sample carry out sample information mark after, obtain figure Piece markup information；Menu picture sample and picture markup information are input in the model of initialization and is trained and tests, Finally obtain algorithm model.Wherein, menu picture sample is true menu picture, and, sample is carried out to menu picture sample The mark of this information is usually to be realized by the way of manually marking, specifically famous to each true menu picture mark The texts such as title, price, unit, specification, classification.

However, this training in the prior art obtains the mode of algorithm model, firstly, the low efficiency manually marked, and The required sample size of training is then very huge, this will cause the time cost manually marked or financial cost high；Secondly, artificial mark Note often means that a large amount of repetitive operation, can bring certain amount of error in this way and be not easy to verify out.

Summary of the invention

In view of the above problems, it proposes on the present invention overcomes the above problem or at least be partially solved in order to provide one kind State recognition methods and the device of the menu information of problem.

According to an aspect of the invention, there is provided a kind of recognition methods of menu information, comprising:

Obtain vegetable text information corresponding with sample vegetable；

Presentation style of the vegetable text information in menu picture is simulated, is generated and the vegetable according to analog result The corresponding simulation menu picture of text information；

The simulation menu picture is determined as menu sample, according to vegetable text information determination and the menu sample This corresponding sample markup information；

Mould is identified according to the menu sample and sample markup information corresponding with menu sample training menu Type obtains the recognition result of menu picture to be identified by the menu identification model.

Optionally, presentation style of the simulation vegetable text information in menu picture includes:

According to preset menu analog rule, presentation style of the vegetable text information in menu picture is simulated；

Wherein, the menu analog rule includes at least one of the following: pattern class sub-rule, and/or scene class Rule.

Optionally, when the menu analog rule includes pattern class sub-rule；

Presentation style of the simulation vegetable text information in menu picture include:

Font style corresponding with the vegetable text information is selected from preset fontlib, according to the font of selection The vegetable text information is presented in pattern in menu picture, obtains analog result corresponding with the font style of selection；With/ Or,

Layout pattern template corresponding with the vegetable text information is selected from preset layout templates library, according to choosing The vegetable text information is presented in the layout pattern template selected in menu picture, obtains opposite with the layout pattern template of selection The analog result answered；Wherein, each text items in the vegetable text information and each text box in layout pattern template It is corresponding.

It optionally, include multiple fonts pattern in the preset fontlib；

Multiple fonts pattern corresponding with the vegetable text information is selected from preset fontlib, according to selection The vegetable text information is presented in multiple fonts pattern in menu picture respectively, obtains opposite with the multiple fonts pattern of selection The multiple analog result answered；

It is described to generate corresponding with vegetable text information simulation menu picture according to analog result and include:

Distinguished according to the various analog results in the multiple analog result corresponding with the multiple fonts pattern of selection Generate simulation menu picture corresponding with the vegetable text information.

It optionally, include a variety of layout pattern templates in the preset layout templates library；

A variety of layout pattern templates corresponding with the vegetable text information are selected from preset layout templates library, are pressed The vegetable text information is presented in menu picture respectively according to a variety of layout pattern templates of selection, obtains a variety of with selection The corresponding multiple analog result of layout pattern template；

According to the various analog results in the multiple analog result corresponding with a variety of layout pattern templates of selection Simulation menu picture corresponding with the vegetable text information is generated respectively.

Optionally, it is described according to the layout pattern template presented in menu picture the vegetable text information it Afterwards, the method also includes:

Determine the offset of each text box in the layout pattern template；The offset includes transversal displacement And/or vertical misalignment amount；

The position of each text box is adjusted according to the offset, the position of each text box after the adjustment, which is presented, to be corresponded to Each text items.

Optionally, when the menu analog rule includes scene class sub-rule；

Determine shadow information when shooting the menu picture；The shadow information is added in the menu picture, is obtained To presentation style of the vegetable text information described under the shadow information in menu picture.

Optionally, when the menu analog rule includes scene class sub-rule；

Presentation style of the simulation vegetable text information in menu picture further include:

Rotation center is determined in the plane where menu picture；And first when shooting is determined according to preset rules Shooting angle；

According to the rotation center and the first shooting angle in each text box of plane internal rotation where menu picture；

Corresponding each text items are presented in the position of each text box after rotation, obtain through the postrotational vegetable Presentation style of the text information in menu picture.

Optionally, when the menu analog rule includes scene class sub-rule；

For the second shooting angle formed between the plane where the imaging plane and menu picture of capture apparatus, calculate The perspective matrix of corresponding second shooting angle；

Perspective transform is carried out to menu picture according to the perspective matrix, obtains the vegetable text after perspective transform Presentation style of the information in menu picture.

Background corresponding with the menu picture is selected from preset context vault, and the background of selection is added to menu The bottom of picture obtains presentation style of the vegetable text information in menu picture under the background of selection.

According to another aspect of the present invention, a kind of identification device of menu information is provided, comprising:

Module is obtained, is suitable for obtaining vegetable text information corresponding with sample vegetable；

Analog module, suitable for simulating presentation style of the vegetable text information in menu picture；

Generation module is suitable for generating simulation menu picture corresponding with the vegetable text information according to analog result；

Determining module, it is true according to the vegetable text information suitable for the simulation menu picture is determined as menu sample Fixed sample markup information corresponding with the menu sample；

Identification module is suitable for being instructed according to the menu sample and sample markup information corresponding with the menu sample Practice menu identification model, the recognition result of menu picture to be identified is obtained by the menu identification model.

Optionally, the analog module is further adapted for:

Optionally, when the menu analog rule includes pattern class sub-rule；

The analog module is further adapted for:

It optionally, include multiple fonts pattern in the preset fontlib；

The analog module is further adapted for:

The generation module is further adapted for:

The analog module is further adapted for:

The generation module is further adapted for:

Optionally, described device further include:

Module is adjusted, is adapted to determine that the offset of each text box in the layout pattern template；The offset packet Include transversal displacement and/or vertical misalignment amount；

And the position suitable for adjusting each text box according to the offset, the position of each text box after the adjustment It sets and corresponding each text items is presented.

Optionally, when the menu analog rule includes scene class sub-rule；

The analog module is further adapted for:

Optionally, when the menu analog rule includes scene class sub-rule；

The analog module is further adapted for:

Optionally, when the menu analog rule includes scene class sub-rule；

The analog module is further adapted for:

Optionally, the analog module is further adapted for:

According to another aspect of the invention, provide a kind of calculating equipment, comprising: processor, memory, communication interface and Communication bus, the processor, the memory and the communication interface complete mutual communication by the communication bus；

For the memory for storing an at least executable instruction, it is above-mentioned that the executable instruction executes the processor The corresponding operation of the recognition methods of menu information.

In accordance with a further aspect of the present invention, provide a kind of computer storage medium, be stored in the storage medium to A few executable instruction, the executable instruction make processor execute the corresponding operation of recognition methods such as above-mentioned menu information.

The recognition methods of menu information according to the present invention and device are getting vegetable text corresponding with sample vegetable After this information, the influence factor of interference is generated to recognition result by simulating, to realize the emulation of true menu, and is emulated Menu picture；Wherein, the vegetable text information for the sample vegetable for including in simulation menu picture be it is known, then can use this Vegetable text information accurately to indicate sample markup information；Simulation menu picture is determined as menu sample, and according to vegetable Text information determines corresponding with menu sample sample markup information, and is trained accordingly to menu identification model, can be with Training obtains accurate menu identification model, then can be used for identifying menu picture to be identified.It can be seen that this hair Bright scheme, directly can accurately determine sample markup information according to vegetable text information, on the one hand, carry out sample mark without artificial The mark for infusing information, saves the time cost manually marked and financial cost；On the other hand, compared to artificial mark, originally Scheme determines the problem of mode of sample markup information avoids marking error, so that the menu identification model that training obtains It is more accurate.

The above description is only an overview of the technical scheme of the present invention, in order to better understand the technical means of the present invention, And it can be implemented in accordance with the contents of the specification, and in order to allow above and other objects of the present invention, feature and advantage can It is clearer and more comprehensible, the followings are specific embodiments of the present invention.

Detailed description of the invention

By reading the following detailed description of the preferred embodiment, various other advantages and benefits are common for this field Technical staff will become clear.The drawings are only for the purpose of illustrating a preferred embodiment, and is not considered as to the present invention Limitation.And throughout the drawings, the same reference numbers will be used to refer to the same parts.In the accompanying drawings:

Fig. 1 shows the flow chart of the recognition methods of menu information according to an embodiment of the invention；

Fig. 2 shows the flow charts of the recognition methods of menu information in accordance with another embodiment of the present invention；

Fig. 3 shows the schematic diagram of the first shooting angle under various standard angles；

Fig. 4 a to Fig. 4 b respectively illustrates the schematic diagram for the menu picture taken under capture apparatus difference inclination conditions；

Fig. 5 shows the functional block diagram of the identification device of menu information according to an embodiment of the invention；

Fig. 6 shows a kind of structural schematic diagram for calculating equipment according to an embodiment of the present invention.

Specific embodiment

Exemplary embodiments of the present disclosure are described in more detail below with reference to accompanying drawings.Although showing the disclosure in attached drawing Exemplary embodiment, it being understood, however, that may be realized in various forms the disclosure without should be by embodiments set forth here It is limited.On the contrary, these embodiments are provided to facilitate a more thoroughly understanding of the present invention, and can be by the scope of the present disclosure It is fully disclosed to those skilled in the art.

Fig. 1 shows the flow chart of the recognition methods of menu information according to an embodiment of the invention.As shown in Figure 1, This method comprises:

Step S101: vegetable text information corresponding with sample vegetable is obtained.

In the present embodiment, by being emulated to obtain simulation menu picture to true menu, and by simulation menu picture It is trained to obtain menu identification model as menu sample.

Wherein, sample vegetable refers to the vegetable for including in simulation menu picture；And vegetable text corresponding with sample vegetable This information includes the content of text for reflecting dish information, and optionally, dish information includes but is not limited to menu name information, vegetable Pricing information, vegetable unit information and/or vegetable category information.

Specifically, vegetable text information corresponding with sample vegetable is actively obtained, that is, is shown in simulation menu picture Vegetable data be it is known, during subsequent mark, can accurately learn the sample vegetable for including in simulation menu picture Sample markup information.Need it is described herein be, in the present embodiment, to vegetable text information corresponding with sample vegetable Source is not specifically limited, and optionally, which can be vegetable library.

Step S102: presentation style of the vegetable text information in menu picture is simulated, according to analog result generation and dish The corresponding simulation menu picture of product text information.

Specifically, presentation style of the simulation vegetable text information in menu picture, is substantially right in simulating actual conditions Identification generates the factor of interference.In a practical situation, presentation style of the vegetable text information in menu picture is mainly by typesetting The influence of pattern and/or shooting environmental, correspondingly, when simulating presentation style of the vegetable text information in menu picture, then Mainly the typesetting pattern and/or actual shooting environmental of vegetable text information in menu picture are simulated.But this hair Bright to be not limited thereto, during specific implementation, those skilled in the art can also be simulated other influence factors, For example, the background of simulation menu picture.

After simulating to presentation style of the vegetable text information in menu picture, the menu map after simulating is obtained Piece generates simulation menu picture corresponding with vegetable text information according to the menu picture after simulating, and the present invention is not yet The mode for generating simulation menu picture is specifically limited, optionally, directly can be determined as imitating by the menu picture after simulating True menu picture, alternatively, can be pre-processed first to the menu picture after simulating, such as picture scaling processing, then Picture after pretreatment is determined as simulation menu picture.

Step S103: being determined as menu sample for simulation menu picture, according to the determination of vegetable text information and menu sample Corresponding sample markup information.

Before training obtains menu identification model, it is first determined for trained menu sample and determination and menu sample This corresponding sample markup information.In the present embodiment, the simulation menu picture actively constructed by emulation mode is determined For menu sample；And the vegetable text information in simulation menu picture including is the normal data got, will emulated Menu picture is determined as after menu sample, then menu text information can accurately be expressed as sample corresponding with menu sample Markup information.As it can be seen that the sample markup information corresponding with menu sample determined in the present embodiment, and true in menu sample Real data is exact matching, and then may make determining sample markup information accurate.

Step S104: mould is identified according to menu sample and sample markup information corresponding with menu sample training menu Type obtains the recognition result of menu picture to be identified by menu identification model.

The sample markup information corresponding with menu sample determined through the above steps, with the true number in menu sample According to exact matching, then menu identification model is trained using menu sample and sample markup information corresponding with menu sample, Can be to avoid because of sample markup information mistake, and the problem of the inaccuracy for the menu identification model for causing training to obtain.Then, sharp The vegetable text information in menu picture to be identified is identified with the menu identification model that training obtains, then it can be accurate The vegetable data identified in menu picture to be identified.

According to the recognition methods of menu information provided in this embodiment, vegetable text corresponding with sample vegetable is being got After this information, the influence factor of interference is generated to recognition result by simulating, to realize the emulation of true menu, and is emulated Menu picture；Wherein, the vegetable text information for the sample vegetable for including in simulation menu picture be it is known, then can use this Vegetable text information accurately to indicate sample markup information；Simulation menu picture is determined as menu sample, and according to vegetable Text information determines corresponding with menu sample sample markup information, and is trained accordingly to menu identification model, can be with Training obtains accurate menu identification model, then can be used for identifying menu picture to be identified.It can be seen that this reality A scheme is applied, sample markup information directly can accurately be determined according to vegetable text information, on the one hand, carries out sample without artificial The mark of markup information saves the time cost manually marked and financial cost；On the other hand, it is marked compared to artificial, This programme determines the problem of mode of sample markup information avoids marking error, so that the menu identification mould that training obtains Type is more accurate.

Fig. 2 shows the flow charts of the recognition methods of menu information in accordance with another embodiment of the present invention.Such as Fig. 2 institute Show, this method comprises:

Step S201: vegetable text information corresponding with sample vegetable is obtained.

Specifically, it according to the demand of practical identification scene, determines the dish information for needing to obtain, makes the dish information obtained It is consistent with the dish information for including in vegetable picture to be identified.Under normal conditions, the vegetable name of vegetable is included at least in menu Claim information and vegetable pricing information, in some alternative embodiments, in menu further includes vegetable unit information and/or vegetable Category information.Then, it obtains each sample vegetable and corresponds to the content of text of dish information to get to corresponding with sample vegetable Vegetable text information.Wherein, the selection of sample vegetable can randomly choose, or can according in practical identification scene the style of cooking or Vegetable type is selected.

For example, it needs menu identification model being applied to the menu that identification is made of menu name and vegetable price, It then determines that dish information is menu name information and vegetable pricing information, and obtains from vegetable library the vegetable of several sample vegetables The content of text of name information and the content of text of vegetable pricing information.

Step S202: according to preset menu analog rule, presentation sample of the vegetable text information in menu picture is simulated Formula；Simulation menu picture corresponding with vegetable text information is generated according to analog result.

Wherein, menu analog rule includes at least one of the following: pattern class sub-rule, and/or scene class sub-rule. Wherein, pattern sub-rule is mainly used for simulating the typesetting pattern of vegetable text information；Scene sub-rule is mainly used for pair Shooting environmental is simulated.Illustrate menu mould from being simulated according to pattern sub-rule and according to scene sub-rule separately below Quasi- detailed process:

First, the presentation style according to pattern sub-rule simulation vegetable text information in menu picture.Pattern sub-rule The font style and/or layout pattern of vegetable text information can be limited.

Specifically, when menu analog rule includes pattern class sub-rule, selection and vegetable text from preset fontlib The corresponding font style of this information；The font style includes font type, font size, font color, and/or font weight Etc. information, wherein font type further includes hand-written script；Vegetable text is presented in menu picture according to the font style of selection Information obtains analog result corresponding with the font style of selection, wherein for a kind of font style of selection, can obtain To a kind of corresponding analog result.It further, include multiple fonts pattern in preset fontlib, due to true dish Single font style be it is diversified, the simulation menu picture to obtain simulation covers more font style scenes, from pre- If fontlib in corresponding with the vegetable text information multiple fonts pattern of selection, according to the multiple fonts pattern difference of selection Vegetable text information is presented in menu picture, obtains multiple analog result corresponding with the multiple fonts pattern of selection；So Afterwards, it is generated respectively according to the various analog results in multiple analog result corresponding with the multiple fonts pattern of selection and vegetable The corresponding simulation menu picture of text information, it can obtain the simulation menu picture of corresponding different fonts pattern.Pass through this Kind mode, can simulate to obtain multiple simulation menu pictures for identical vegetable text information, and then can cover more The possibility of font style is conducive to the identification that the menu identification model for obtaining training is suitable for various font style scenes, excellent Change recognition effect.

And/or when menu analog rule includes pattern class sub-rule, selection and vegetable from preset layout templates library The corresponding layout pattern template of text information, wherein every vegetable in layout pattern template including same sample vegetable is believed The layout of the content of text of breath, for example, the layout of menu name and vegetable price；Dish information between different sample vegetables Layout, for example, menu name is horizontally-arranged layout or vertical setting of types layout；And/or the layout of the vegetable picture including sample vegetable.Base In this, in the specific example that dish information includes menu name information and vegetable pricing information, following layout parameter can be passed through One of or it is a variety of come definition and layout pattern template: the relative position between menu name information and vegetable pricing information, dish The writing direction of product name information, the quantity of the vegetable pricing information of same sample vegetable and same sample vegetable it is multiple Relative position between vegetable pricing information.In the specific implementation, those skilled in the art may deposit in combination with true menu Layout pattern corresponding template be set and store into layout templates library, optionally, layout pattern template includes standard cross Row, standard vertical setting of types, multivalence lattice are horizontally-arranged, day Korean style vertical setting of types and/or with figure typesetting, alternatively, being also possible that class in true menu Mesh information, then setting is stored comprising the template of category information into layout templates library.Then, according to the layout pattern template of selection Vegetable text information is presented in menu picture, obtains analog result corresponding with the layout pattern template of selection；Wherein, dish Each text items in product text information are corresponding with each text box in layout pattern template.Wherein, in layout pattern template Layout parameter is provided with multiple text boxes, it, only need to be by items vegetable letter corresponding in vegetable text information when being simulated In content of text (i.e. text items) the filling text box of breath.It further, include a variety of cloth in preset layout templates library Office's pattern template, also due to the layout pattern of true menu be it is diversified, the simulation menu picture to obtain simulation covers More layout pattern scenes are covered, a variety of layout samples corresponding with vegetable text information are selected from preset layout templates library Formula template is presented vegetable text information according to a variety of layout pattern templates of selection in menu picture respectively, obtains and select The corresponding multiple analog result of a variety of layout pattern templates；Then, according to opposite with a variety of layout pattern templates of selection The various analog results in multiple analog result answered generate simulation menu picture corresponding with vegetable text information respectively, i.e., The simulation menu picture of the different layout pattern templates of available correspondence.In this way, for identical vegetable text envelope Breath can simulate to obtain multiple simulation menu pictures, and then can cover the possibility of more layout patterns, be conducive to make to train Obtained menu identification model is suitable for the scene of various layout patterns, Statistical error effect.

Further, after simulating to layout pattern, what is obtained is arranged according to the layout pattern template of standard The simulation menu picture of version, at this point, determining the offset of each text box in layout pattern template；Offset includes laterally inclined Shifting amount and/or vertical misalignment amount；The position of each text box is adjusted according to offset, the position of each text box after the adjustment Corresponding each text items are presented.In practice, true menu can be because artificial design or adjustment, lead to each text items not It can be presented in strict accordance with each text box in the layout pattern template of standard, for example, in left-justify, Right Aligns or placed in the middle right In neat layout format, the difference of the character length of vegetable text information, it is most likely that cause to cannot achieve complete left-justify, Right Aligns or align center adjust the position of each text box in the layout pattern template of selection, make this for this scene Text box shifts some or all of in layout pattern template, and each text items are presented on each text adjusted In frame, that is, realize the simulation to the scene that can not be absolutely aligned in true menu.

In the present invention, the concrete mode for determining the offset of each text box is not limited, can determine, can also pass through at random Algorithm determines.In some alternative embodiments, the offset of each text box can be determined as follows: being primarily based on general Rate distribution function ro (lambda, sigma) determines the transversal displacement of each text box.Wherein, lambda indicates row coordinate Base position, sigma be its irrelevance degree；And in the optional embodiment, the tool of probability-distribution function is not limited Body type comprising but be not limited to following any: normal distribution is uniformly distributed, exponential distribution and Poisson distribution；Also, Parameter needed for determining probability-distribution function can also flexibly be set, and by taking normal distribution as an example, settable mean value is 0, and variance is One character length.After probability-distribution function has been determined, the inclined of each text box is simulated using Monte Carlo sampling Shifting amount.By generating random number, when the value of random number is less than the value of probability-distribution function, then the point is selected otherwise not select This point；Extract the point selected out its coordinate position one by one according to probability-distribution function.Come secondly based on probability-distribution function true The vertical misalignment amount of fixed each text box, determination process can also refer to the tool of the transversal displacement of each text box of above-mentioned determination Body process, details are not described herein again.

In addition, for presentation sample of the vegetable text information of unified simulation in menu picture is carried out according to pattern sub-rule Formula, the mode for being referred to the position of above-mentioned adjustment text box are adjusted for each text box and its text items of presentation It is whole, the not quite identical situation of presentation style to simulate vegetable text information in true menu.For example, in the word according to selection After body pattern simulates the font style of vegetable text information, it may be based on probability-distribution function and adjust part of text items Font size.

Second, the presentation style according to scene sub-rule simulation vegetable text information in menu picture.Scene sub-rule The shadow and/or angle of shooting can be limited.

Specifically, when menu analog rule includes scene class sub-rule, shadow information when shooting menu picture is determined； Wherein, bright and dark light when shadow information is primarily referred to as reflection shooting, with the presence or absence of blocking, and/or blocking position and size Information；Due to the complicated multiplicity of actual shooting environmental, often there is a situation where to block or light is bad, can then make at this time The menu picture that must be shot is there are shade, in the present embodiment, determines the shadow information under corresponding actual shooting environmental, example Such as, it for the shade for blocking generation of finger in shooting process, can be simulated, that is, determined by the fillet quadrangle of certain luminance Shadow information is the fillet quadrangle of certain luminance, wherein certain luminance can be simulated with quantum condition entropy.Then in menu picture Middle addition shadow information, obtains presentation style of the vegetable text information in menu picture under shadow information.Also, specific In implementation process, it will be assumed that shadow condition caused by under a large amount of different shooting environmentals, and determine corresponding shadow information, with needle The simulation menu picture of corresponding different shadow information is simulated to identical vegetable text information, so that final trained To menu identification model be applicable to the identification under the scene of various shadow information, Statistical error effect.

And/or when menu analog rule includes scene class sub-rule, rotation is determined in the plane where menu picture Center；And the first shooting angle when shooting is determined according to preset rules, wherein rotation center can be menu picture institute Plane in any point, optionally, rotation center be menu picture center；And first shooting angle refer to when shooting When the imaging plane of equipment is parallel with the plane where menu picture, the central axis of imaging plane and the central axis of menu picture Angle between line.According to rotation center and the first shooting angle in each text of plane internal rotation where menu picture Frame；Corresponding each text items are presented in the position of each text box after rotation, obtain through postrotational vegetable text information Presentation style in menu picture.Drift angle clockwise or counterclockwise is inevitably had when actual photographed menu, so that clapping In the menu picture taken the photograph the vegetable text information of a row or column not with the sides aligned parallel of menu picture, in the present embodiment In, then menu picture is rotated by the rotation transformation in affine transformation, is deposited with simulating the menu picture that actual photographed goes out The inclination angle the case where.Further, standard angle s when shooting be usually 0 degree, 90 degree, 180 degree or 270 degree, and by first Shooting angle is set as r, indicates low-angle that is counterclockwise at standard angle s or rotating clockwise, that is, the actual photographed simulated When inclination angle, then the angle of the simulation menu picture finally simulated is a=s (+/-) r.Wherein, the first shooting angle r can be with It is simulated with probability-distribution function, likewise, also not limiting the concrete type of probability-distribution function herein comprising but be not limited to Any below: normal distribution is uniformly distributed, exponential distribution and Poisson distribution.Fig. 3 is shown under various standard angles The schematic diagram of first shooting angle.As shown in figure 3, using the positive direction of the longitudinal axis as 0 degree of standard angle of direction, the negative direction of the longitudinal axis The negative direction in the direction of standard angle 180 degree, direction and horizontal axis that the positive direction of horizontal axis is 270 degree of standard angle is standard The direction that 90 degree of angle, then to counterclockwise or being rotated clockwise the on the basis of b and b ' is illustrated respectively in 0 degree of standard angle One shooting angle；Similarly, c and c ', d and d ' and a and a ' are illustrated respectively on the basis of corresponding standard angle to counterclockwise Or it is rotated clockwise the first shooting angle.Further, by the way that menu picture is rotated about the center of rotation different One shooting angle can be simulated to obtain simulation menu picture when corresponding to different shooting inclination angles, so that final training obtains Menu identification model be applicable to it is various shooting inclination angle scenes under identifications, Statistical error effect.

And/or when menu analog rule includes scene class sub-rule, for the imaging plane and menu map of capture apparatus Formed second shooting angle between plane where piece, calculates the perspective matrix of corresponding second shooting angle；According to perspective square Battle array carries out perspective transform to menu picture, obtains presentation sample of the vegetable text information after perspective transform in menu picture Formula.When actual photographed menu, capture apparatus usually there will be the certain elevation angle or the angle of depression compared to the plane where menu picture, The elevation angle or the angle of depression are formed second shooting angle between the imaging plane of capture apparatus and the plane where menu picture, It will cause the width of all directions for the menu picture shot is inconsistent at this time.Fig. 4 a to Fig. 4 b respectively illustrates shooting and sets The schematic diagram for the menu picture taken under standby different inclination conditions.As shown in Figs. 4a and 4b, due to the imaging plane of capture apparatus It is not parallel with the plane where menu ABCD, cause the menu picture A ' B ' C ' D ' shot occur up-narrow and down-wide in Fig. 4 a or There is the left wide right narrow situation in Fig. 4 b in person.For above-mentioned actual shooting situation, for the second different shooting angle, benefit Perspective transform is carried out to menu picture with corresponding perspective matrix, when being shot with simulating with the second shooting angle, is shot The menu picture arrived.Further, perspective transform is carried out to menu picture by multiple perspective matrix, can simulates and is corresponded to Simulation menu picture when the difference shooting elevation angle or the angle of depression, so that the menu identification model that final training obtains is applicable to Identification under the various shooting elevations angle or angle of depression scene, Statistical error effect.

For convenient for the understanding to above-mentioned perspective transform, for following figure 4 b, to illustrate a kind of specific implementation of perspective transform Process:

The original vertices on the vertex of menu picture ABCD, then original vertices coordinate are as follows: X=[(xa, ya), (xb, yb), (xc, yc), (xd, yd)], and the vertex for the menu picture A ' B ' C ' D ' that shooting obtains is set as perspective vertex, then have an X-rayed top Point coordinate are as follows: and Y=[(xa ', ya '), (xb ', yb '), (xc ', yc '), (xd ', yd ')].

If perspective second-order matrix is P, P then meets: Y=P*X+k*I, and, it is convenience of calculation, it is assumed that | P |=1, still Truth is not limited to this, and P has 4 parameters, can construct 4 equations by 4 apex coordinates, in addition | P |=1 Assuming that the equation with k*I can be solved, non trivial solution can be so found out, this mode can solve any type of perspective and become It changes, specific calculating process is no longer described in detail.

In fig. 4b, vertex meets the following conditions:

A=A '；

D=D '；

Xb=xb '-c；

Yb=yb '+d；

Xc=xc '-c；

Yc=yc '-d；

Wherein c indicates the amount of bias of horizontal direction, and d indicates the amount of bias of vertical direction, can also pass through ratio value meter herein It calculates, but addition and subtraction calculation amount is smaller, performance is more preferable.

Perspective matrix P can be calculated by above-mentioned definition, can determine that each point is being had an X-rayed in original image according to P Position in image.

Need it is described herein be, although the one in step S202 and secondly two o'clock only respectively from according to pattern sub-rule Illustrate the detailed process of menu simulation with being simulated according to scene sub-rule, but the present invention is not limited thereto, specifically When implementation, the influence factor that those skilled in the art can also generate interference for other pairs of identifications is simulated, so that training Obtained menu identification model can promote recognition effect suitable for more various situation.Optionally, of the invention one In a specific embodiment, the background of menu picture can be simulated, be selected from preset context vault opposite with menu picture The background of selection is added to the bottom of menu picture by the background answered, and obtains under the background of selection vegetable text information in dish Presentation style in free hand drawing piece.Further, also include a variety of backgrounds in preset context vault, selected from preset context vault A variety of menu backgrounds are presented vegetable text information according to a variety of menu backgrounds of selection in menu picture respectively, obtain and select The corresponding multiple analog result of a variety of menu backgrounds selected；Then, according to corresponding more with a variety of menu backgrounds of selection Various analog results in kind analog result generate simulation menu picture corresponding with vegetable text information respectively, it can To the simulation menu picture of corresponding different menu background.In addition, needing when selecting menu background by background color and font face Color is staggered, and the two color is avoided to be too close to cause human eye that can not all identify vegetable text information.

In addition, in the present embodiment, not limiting the simulation order of above-mentioned a variety of real scenes, it is preferred that can be first according to sample Formula rule simulates typesetting pattern, simulates shooting environmental according still further to scene sub-rule.And in some alternative embodiments, The simulation of above-mentioned part real scene can also suitably be reduced；Alternatively, can also suitably increase in other optional embodiments Add and the influence factor interfered is generated to identification present in unmentioned in other present invention and true menu or shooting environmental Simulation.

By step S202, magnanimity simulation menu picture, these simulation menus can be quickly obtained by way of simulation Picture covers the various problems being likely to occur in real scene, and, it further include a variety of completely new in these simulation menu pictures Menu.

Step S203: being determined as menu sample for simulation menu picture, according to the determination of vegetable text information and menu sample Corresponding sample markup information.

Using the menu picture of necessary being as menu sample in compared to the prior art, by simulation menu in the present embodiment Picture is determined as menu sample, not only can with the menu sample of quick obtaining to magnanimity, but also these menu samples covering field Scape is extensive, there is a large amount of completely new menu, so that the sample set of menu sample more horn of plenty.

And vegetable text information is determined as sample markup information corresponding with menu sample.Wherein, if vegetable is literary This information includes the corresponding content of text of multinomial dish information, when determining sample markup information corresponding with menu sample, Further the corresponding content of text of multinomial dish information of each sample vegetable can be matched according to preset mark sequence, It obtains corresponding to the markup information of each sample vegetable, then the markup information of the multiple sample vegetables of correspondence is integrated, is obtained Sample markup information corresponding with menu sample；Wherein, preset mark sequence refers to the mark of preset multinomial dish information Note sequence, the preset mark sequence can be configured according to the application scenarios of recognition result.By by multinomial dish information pair The content of text answered can make to train obtained menu identification model in such a way that preset mark sequence carries out pairing mark It is used not only for identifying the word content in menu, and the corresponding relationship of word content can be matched, in turn Data handling procedure after can simplify identification.

By taking dish information includes menu name information and vegetable pricing information as an example, and in the application scenarios of recognition result, It needs as unit of vegetable, according to recognition result successively typing menu name and vegetable price, then preset mark sequence is dish Product name information is preceding, and vegetable pricing information is rear.Assuming that including 3 sample vegetables, respectively sample vegetable in menu sample 1, sample vegetable 2 and sample vegetable 3, correspondingly, in the process for determining sample markup information corresponding with menu sample In, first respectively by the vegetable text information of each sample vegetable according to menu name information preceding, vegetable pricing information is rear Sequence matched, 3 markup informations for obtaining corresponding 3 sample vegetables are respectively (menu name 1, vegetable price 1), (menu name 2, vegetable price 2), and (menu name 3, vegetable price 3),；Then 3 markup informations integrate To sample markup information corresponding with menu sample are as follows: { (menu name 1, vegetable price 1), (menu name 2, vegetable price 2), (menu name 3, vegetable price 3) }.

Also, the vegetable text information for including in simulation menu picture is the normal data got, will emulate dish Free hand drawing piece is determined as after menu sample, then menu text information can accurately be expressed as sample mark corresponding with menu sample Information is infused, and then the problem of be not in marking error.

Step S204: mould is identified according to menu sample and sample markup information corresponding with menu sample training menu Type obtains the recognition result of menu picture to be identified by menu identification model.

This embodiment scheme can be widely applied in the various scenes for needing to obtain the content of text in menu, below with Menu information typing scene illustrates the effect of this embodiment scheme: in previous food product information upload procedure, requiring quotient Family is by uploading food product picture, food product title, characteristic introduction, the information such as price, due to the food product library of businessman to store it is various each The food product of sample, upload quantity is also very huge, if per pass food product takes time and effort all in accordance with such upload mode, and passes through This programme is trained to obtain menu identification model using the magnanimity simulation menu picture that emulation obtains, and then carries out menu letter Typing is ceased, intelligence may be implemented and record dish；Also, in training sample menu quantity enough, menu covering scene domain it is wide, in turn The menu identification model that training can be made to obtain is more accurate and can be applied to the record of the menu information under various complex scenes Enter.

According to the recognition methods of menu information provided in this embodiment, vegetable text corresponding with sample vegetable is being got After this information, the influence factor of interference is generated to recognition result by simulating, obtains simulation menu picture, it is this by emulating To the scheme of the menu sample for training, the menu sample of magnanimity not only can be quickly obtained, but also these menu samples cover The scene of lid is extensive, there is a large amount of completely new menu, so that the sample set of menu sample more horn of plenty, correspondingly, It is trained using the menu sample of the magnanimity, then the available menu identification model that it applies more widely, Jin Eryou Conducive to Statistical error effect.And simulation menu picture be actively construct, it includes sample vegetable vegetable text information Be it is known, then can use the vegetable text information accurately to indicate sample markup information, and avoid asking for marking error Topic.It can be seen that this embodiment scheme, directly can accurately determine sample markup information according to vegetable text information, on the one hand, Without manually carrying out the mark of sample markup information, the time cost manually marked and financial cost are saved；On the other hand, Compared to artificial mark, this programme determines the problem of mode of sample markup information avoids marking error, so that training Obtained menu identification model is more accurate；And it is possible to obtaining magnanimity by emulation mode and the various scenes of covering imitative True menu picture, and then the menu identification model that training can be made to obtain is suitable for the identification scene of various complexity.

Fig. 5 shows the functional block diagram of the identification device of menu information according to an embodiment of the invention.Such as Fig. 5 institute Show, which includes:

Module 501 is obtained, is suitable for obtaining vegetable text information corresponding with sample vegetable；

Analog module 502, suitable for simulating presentation style of the vegetable text information in menu picture；

Generation module 503 is suitable for generating simulation menu figure corresponding with the vegetable text information according to analog result Piece；

Determining module 504, suitable for the simulation menu picture is determined as menu sample, according to the vegetable text information Determine sample markup information corresponding with the menu sample；

Identification module 505 is suitable for according to the menu sample and sample corresponding with menu sample mark letter Training menu identification model is ceased, the recognition result of menu picture to be identified is obtained by the menu identification model.

In a kind of optional embodiment, the analog module 502 is further adapted for:

In a kind of optional embodiment, when the menu analog rule includes pattern class sub-rule；

The analog module 502 is further adapted for:

It include multiple fonts pattern in the preset fontlib in a kind of optional embodiment；

The analog module 502 is further adapted for:

The generation module 503 is further adapted for:

It include a variety of layout pattern templates in the preset layout templates library in a kind of optional embodiment；

The analog module 502 is further adapted for:

The generation module 503 is further adapted for:

In a kind of optional embodiment, described device further include:

Module 506 is adjusted, is adapted to determine that the offset of each text box in the layout pattern template；The offset Including transversal displacement and/or vertical misalignment amount；

In a kind of optional embodiment, when the menu analog rule includes scene class sub-rule；

The analog module 502 is further adapted for:

In a kind of optional embodiment, the analog module 502 is further adapted for:

It can refer to the description of corresponding steps in embodiment of the method about the specific structure and working principle of above-mentioned modules, Details are not described herein again.

The embodiment of the present application provides a kind of nonvolatile computer storage media, and the computer storage medium is stored with The identification of the menu information in above-mentioned any means embodiment can be performed in an at least executable instruction, the computer executable instructions Method.

Fig. 6 shows a kind of structural schematic diagram for calculating equipment according to an embodiment of the present invention, the specific embodiment of the invention The specific implementation for calculating equipment is not limited.

As shown in fig. 6, the calculating equipment may include: processor (processor) 602, communication interface (Communications Interface) 604, memory (memory) 606 and communication bus 608.

Wherein:

Processor 602, communication interface 604 and memory 606 complete mutual communication by communication bus 608.

Communication interface 604, for being communicated with the network element of other equipment such as client or other servers etc..

Processor 602 can specifically execute in the recognition methods embodiment of above-mentioned menu information for executing program 610 Correlation step.

Specifically, program 610 may include program code, which includes computer operation instruction.

Processor 602 may be central processor CPU or specific integrated circuit ASIC (Application Specific Integrated Circuit), or be arranged to implement the integrated electricity of one or more of the embodiment of the present invention Road.The one or more processors that equipment includes are calculated, can be same type of processor, such as one or more CPU；It can also To be different types of processor, such as one or more CPU and one or more ASIC.

Memory 606, for storing program 610.Memory 606 may include high speed RAM memory, it is also possible to further include Nonvolatile memory (non-volatile memory), for example, at least a magnetic disk storage.

Program 610 specifically can be used for so that processor 602 executes following operation:

Obtain vegetable text information corresponding with sample vegetable；

In a kind of optional embodiment, program 610 can specifically be further used for so that processor 602 execute it is following Operation:

Program 610 can specifically be further used for so that processor 602 executes following operation: selecting from preset fontlib Font style corresponding with the vegetable text information is selected, the dish is presented in menu picture according to the font style of selection Product text information obtains analog result corresponding with the font style of selection；And/or

Program 610 can specifically be further used for so that processor 602 executes following operation: selecting from preset fontlib Multiple fonts pattern corresponding with the vegetable text information is selected, according to the multiple fonts pattern of selection respectively in menu picture It is middle that the vegetable text information is presented, obtain multiple analog result corresponding with the multiple fonts pattern of selection；

Program 610 can specifically be further used for so that processor 602 executes following operation: from preset layout templates library It is middle to select a variety of layout pattern templates corresponding with the vegetable text information, according to a variety of layout pattern templates point of selection The vegetable text information is not presented in menu picture, obtains a variety of moulds corresponding with a variety of layout pattern templates of selection Quasi- result；

In a kind of optional embodiment, program 610 can specifically be further used for so that processor 602 execute it is following Operation: the offset of each text box in the layout pattern template is determined；The offset include transversal displacement and/or Vertical misalignment amount；

Program 610 can specifically be further used for so that processor 602 executes following operation: determining and shoot the menu map Shadow information when piece；The shadow information is added in the menu picture, obtains the vegetable under the shadow information Presentation style of the text information in menu picture.

Program 610 can specifically be further used for so that processor 602 executes following operation: flat where menu picture Rotation center is determined in face；And the first shooting angle when shooting is determined according to preset rules；

Program 610 can specifically be further used for so that processor 602 executes following operation: for the imaging of capture apparatus Formed second shooting angle between plane where plane and menu picture, calculates the perspective of corresponding second shooting angle Matrix；

Algorithm and display are not inherently related to any particular computer, virtual system, or other device provided herein. Various general-purpose systems can also be used together with teachings based herein.As described above, it constructs required by this kind of system Structure be obvious.In addition, the present invention is also not directed to any particular programming language.It should be understood that can use various Programming language realizes summary of the invention described herein, and the description done above to language-specific is to disclose this hair Bright preferred forms.

In the instructions provided here, numerous specific details are set forth.It is to be appreciated, however, that implementation of the invention Example can be practiced without these specific details.In some instances, well known method, structure is not been shown in detail And technology, so as not to obscure the understanding of this specification.

Similarly, it should be understood that in order to simplify the disclosure and help to understand one or more of the various inventive aspects, Above in the description of exemplary embodiment of the present invention, each feature of the invention is grouped together into single implementation sometimes In example, figure or descriptions thereof.However, the disclosed method should not be interpreted as reflecting the following intention: i.e. required to protect Shield the present invention claims features more more than feature expressly recited in each claim.More precisely, as following Claims reflect as, inventive aspect is all features less than single embodiment disclosed above.Therefore, Thus the claims for following specific embodiment are expressly incorporated in the specific embodiment, wherein each claim itself All as a separate embodiment of the present invention.

Those skilled in the art will understand that can be carried out adaptively to the module in the equipment in embodiment Change and they are arranged in one or more devices different from this embodiment.It can be the module or list in embodiment Member or component are combined into a module or unit or component, and furthermore they can be divided into multiple submodule or subelement or Sub-component.Other than such feature and/or at least some of process or unit exclude each other, it can use any Combination is to all features disclosed in this specification (including adjoint claim, abstract and attached drawing) and so disclosed All process or units of what method or apparatus are combined.Unless expressly stated otherwise, this specification is (including adjoint power Benefit require, abstract and attached drawing) disclosed in each feature can carry out generation with an alternative feature that provides the same, equivalent, or similar purpose It replaces.

In addition, it will be appreciated by those of skill in the art that although some embodiments described herein include other embodiments In included certain features rather than other feature, but the combination of the feature of different embodiments mean it is of the invention Within the scope of and form different embodiments.For example, in the following claims, embodiment claimed is appointed Meaning one of can in any combination mode come using.

Various component embodiments of the invention can be implemented in hardware, or to run on one or more processors Software module realize, or be implemented in a combination thereof.It will be understood by those of skill in the art that can be used in practice In the identification device of microprocessor or digital signal processor (DSP) to realize menu information according to an embodiment of the present invention The some or all functions of some or all components.The present invention is also implemented as executing method as described herein Some or all device or device programs (for example, computer program and computer program product).Such reality Existing program of the invention can store on a computer-readable medium, or may be in the form of one or more signals. Such signal can be downloaded from an internet website to obtain, and perhaps be provided on the carrier signal or in any other forms It provides.

It should be noted that the above-mentioned embodiments illustrate rather than limit the invention, and ability Field technique personnel can be designed alternative embodiment without departing from the scope of the appended claims.In the claims, Any reference symbol between parentheses should not be configured to limitations on claims.Word "comprising" does not exclude the presence of not Element or step listed in the claims.Word "a" or "an" located in front of the element does not exclude the presence of multiple such Element.The present invention can be by means of including the hardware of several different elements and being come by means of properly programmed computer real It is existing.In the unit claims listing several devices, several in these devices can be through the same hardware branch To embody.The use of word first, second, and third does not indicate any sequence.These words can be explained and be run after fame Claim.

Claims

1. a kind of recognition methods of menu information, comprising:

Obtain vegetable text information corresponding with sample vegetable；

Presentation style of the vegetable text information in menu picture is simulated, is generated and the vegetable text according to analog result The corresponding simulation menu picture of information；

The simulation menu picture is determined as menu sample, according to vegetable text information determination and the menu sample phase Corresponding sample markup information；

According to the menu sample and sample markup information corresponding with menu sample training menu identification model, lead to Cross the recognition result that the menu identification model obtains menu picture to be identified.

2. according to the method described in claim 1, wherein, presentation of the simulation vegetable text information in menu picture Pattern includes:

Wherein, the menu analog rule includes at least one of the following: pattern class sub-rule, and/or scene class sub-rule.

3. according to the method described in claim 2, wherein, when the menu analog rule includes pattern class sub-rule；

Font style corresponding with the vegetable text information is selected from preset fontlib, according to the font style of selection The vegetable text information is presented in menu picture, obtains analog result corresponding with the font style of selection；And/or

Layout pattern template corresponding with the vegetable text information is selected from preset layout templates library, according to selection The vegetable text information is presented in layout pattern template in menu picture, obtains corresponding with the layout pattern template of selection Analog result；Wherein, each text items in the vegetable text information are corresponding with each text box in layout pattern template.

4. according to the method described in claim 3, including multiple fonts pattern in the preset fontlib wherein；

Multiple fonts pattern corresponding with the vegetable text information is selected from preset fontlib, according to a variety of of selection The vegetable text information is presented in font style in menu picture respectively, obtains corresponding with the multiple fonts pattern of selection Multiple analog result；

It is generated respectively according to the various analog results in the multiple analog result corresponding with the multiple fonts pattern of selection Simulation menu picture corresponding with the vegetable text information.

5. the method according to claim 3 or 4, wherein include a variety of layout patterns in the preset layout templates library Template；

A variety of layout pattern templates corresponding with the vegetable text information are selected from preset layout templates library, according to choosing The vegetable text information is presented in a variety of layout pattern templates selected in menu picture respectively, obtains a variety of layouts with selection The corresponding multiple analog result of pattern template；

Distinguished according to the various analog results in the multiple analog result corresponding with a variety of layout pattern templates of selection Generate simulation menu picture corresponding with the vegetable text information.

6. according to the described in any item methods of claim 3-5, wherein it is described according to the layout pattern template in menu map After the vegetable text information is presented in piece, the method also includes:

Determine the offset of each text box in the layout pattern template；The offset include transversal displacement and/or Vertical misalignment amount；

The position of each text box is adjusted according to the offset, the position of each text box after the adjustment presents corresponding each A text items.

7. according to the described in any item methods of claim 2-6, wherein when the menu analog rule includes scene class sub-rule When；

Determine shadow information when shooting the menu picture；The shadow information is added in the menu picture, is obtained Presentation style of the vegetable text information in menu picture under the shadow information.

8. a kind of identification device of menu information, comprising:

Determining module, suitable for the simulation menu picture is determined as menu sample, according to the vegetable text information determine with The corresponding sample markup information of the menu sample；

Identification module is suitable for according to the menu sample and sample markup information corresponding with menu sample training dish Single identification model obtains the recognition result of menu picture to be identified by the menu identification model.

9. a kind of calculating equipment, comprising: processor, memory, communication interface and communication bus, the processor, the storage Device and the communication interface complete mutual communication by the communication bus；

The memory executes the processor as right is wanted for storing an at least executable instruction, the executable instruction Ask the corresponding operation of the recognition methods of menu information described in any one of 1-7.

10. a kind of computer storage medium, an at least executable instruction, the executable instruction are stored in the storage medium Processor is set to execute the corresponding operation of recognition methods such as menu information of any of claims 1-7.