CN113298072A

CN113298072A - Method and device for identifying design draft and electronic equipment

Info

Publication number: CN113298072A
Application number: CN202010108924.9A
Authority: CN
Inventors: 刘一奇
Original assignee: Alibaba Group Holding Ltd
Current assignee: Alibaba Group Holding Ltd
Priority date: 2020-02-21
Filing date: 2020-02-21
Publication date: 2021-08-24

Abstract

The embodiment of the specification provides a method and a device for identifying a design draft and electronic equipment, wherein the method comprises the following steps: acquiring a design draft provided by a user, wherein the design draft is in a picture format; identifying at least one first component in the design; selecting a first component for which further identification is to be performed; and identifying, in the selected first component, at least one second component.

Description

Method and device for identifying design draft and electronic equipment

Technical Field

The present disclosure relates to the technical field of page design, and in particular, to a method for recognizing a design draft, a method and an apparatus for automatically generating a webpage, and an electronic device.

Background

When the front end webpage page is designed, a design draft can be used. The design draft is a draft that can be used for page design, for example, pictures in JPEG, PNG format. The computer can automatically recognize the elements in the design draft as the components and automatically generate corresponding codes, so that the interface design is realized.

Therefore, it is necessary to provide a scheme to reduce the recognition difficulty in the recognition process of the design.

Disclosure of Invention

According to a first aspect of the present specification, there is provided a method of recognizing a design, comprising: acquiring a design draft provided by a user, wherein the design draft is in a picture format; identifying at least one first component in the design; selecting a first component for which further identification is to be performed; and identifying, in the selected first component, at least one second component.

According to a second aspect of the present specification, there is provided a method of automatically generating a web page, comprising: acquiring a webpage design draft provided by a user, wherein the webpage design draft is in a picture format; identifying at least one first component in the webpage design; selecting a first component for which further identification is to be performed; in the selected first component, at least one second component is identified, and elements on the web page are automatically generated based on the identified first and second components.

According to a third aspect of the present specification, there is provided an apparatus for recognizing a design, comprising: the acquisition module acquires a design draft provided by a user, wherein the design draft is in a picture format; the first identification module identifies at least one first component in the design draft; a selection module that selects a first component for which further identification is to be performed; and a second identifying module that identifies at least one second component among the selected first components.

According to a fourth aspect of the present description, there is provided an electronic device comprising a processor and a memory, the memory storing executable instructions that, when the electronic device is operated, control the processor to perform a method according to an embodiment.

According to a fourth aspect of the present description, there is provided a storage medium storing executable instructions that when executed implement a method according to an embodiment.

In different embodiments, in the process of recognizing the design draft, the larger components are recognized firstly, and then the smaller components are recognized, so that the processing amount caused by the recognition processing of the components with different sizes is reduced, and the overall recognition difficulty is reduced.

Other features of embodiments of the present specification and advantages thereof will become apparent from the following detailed description of exemplary embodiments thereof, which proceeds with reference to the accompanying drawings.

Drawings

The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate embodiments and together with the description, serve to explain the principles of the various embodiments.

FIG. 1 shows a schematic flow diagram of a method of identifying a design according to one embodiment.

FIG. 2 shows a schematic flow chart diagram of a method of automatically generating a web page according to one embodiment.

FIG. 3 shows a schematic block diagram of an apparatus to recognize a design according to one embodiment.

FIG. 4 shows a schematic block diagram of an electronic device according to one embodiment.

Fig. 5-8 illustrate an exemplary example of identifying a design.

Fig. 9 shows a scenario in which a plurality of users collaboratively use a design to design a front page in a network scenario.

Fig. 10 shows an example of the draft collaborative design system.

Detailed Description

The following description of at least one exemplary embodiment is merely illustrative in nature and is in no way intended to limit the invention, its application, or uses.

When designing a web page, a designer may draw a desired interface element, such as a text box, a video box, a control, etc., in a design script. The designer can draw the design draft manually by using a painting brush, and can also draw the design draft by using drawing software. The designer then enters the drawn design into a computer. The computer may automatically identify the components in the design draft and generate corresponding code. The code here may be a schematized code. The developer may later modify the code as needed to achieve the desired functionality. A design used in front-end web page design may be referred to as a web page design. In addition, in the interface design of other application software, the designer can be helped to design the interface of the application software by the aid of design draft.

In the process of identifying the design by the computer, the computer scans all elements in the design and identifies the elements to produce corresponding components. When a computer performs scanning, a larger component and a smaller component need to be processed simultaneously, and the method occupies more computing resources. In addition, during the process of the computer scanning all the components at once, the nesting relation information between the components can be lost.

Therefore, a new scheme for recognizing a design is proposed. In the following, different embodiments and examples of the present description are described with reference to the drawings.

FIG. 1 shows a schematic flow diagram of a method of identifying a design according to one embodiment. The designer can draw the design draft manually by using a painting brush, and can also draw the design draft by using drawing software in a computer. In the case of manual drawing, the drawn design may be input into a computer by a scanner or a camera. In the case of drawing software, after a designer draws a design, he may save the design to a computer. Then, processing is performed by the computer to identify the design. Fig. 1 shows the steps of a method that can be performed in a computer.

As shown in fig. 1, in step S102, a design draft provided by a user is acquired. The design is in a picture format, e.g., JPEG, PNG, etc.

At step S104, at least one first component in the design is identified.

For example, a first outline of the first component may be determined based on a color change in the design, and then the first component may be trimmed from the design along the first outline. The cropped area can be directly used as the identified first component; the recognition of the first component may also be accomplished by continuing to recognize content, e.g., text, controls, etc., in the cropped region, depending on the system configuration.

In step S106, a first component to be further identified is selected. All the first components can be selected for re-identification, and part of the first components can be selected for identification. For example, some first components that no longer contain other elements, such as lines, text, etc., may no longer be identified further on such first components.

In step S108, among the selected first components, at least one second component is identified.

For example, a second outline of a second component may be determined based on a color change in the first component, and then the first component may be cut from the design document along the second outline.

Although one identification process is described herein with respect to a first component and a second component, it will be understood by those skilled in the art that the method described above may be continued with the current second component being the first component in the next identification process.

Here, the larger component is identified first, and then the smaller component is identified, without the need to identify the smaller component at once. The smaller component cropping process may be identified. This may reduce the difficulty of identifying smaller components.

The design may include a plurality of first components. The plurality of first components may be separate from each other or may partially overlap. In addition, the design may also include a plurality of second components, which may also be separate from each other or may partially overlap.

In addition, the nested structure of the design draft can be conveniently reserved by the hierarchical identification mode. For example, a nesting relationship of a first component and a second component may be recorded based on an identified order of the first component and the second component. Therefore, the nested structure of the design draft can be naturally generated in the identification process, so that the processing efficiency is improved, and the processing resource is saved.

In addition, in this way, a higher degree of freedom can also be provided for the recognition process. Different identification modes can be adopted for components of different levels. For example, a larger first component may be used only to divide different functional areas in a page, with a smaller component including an annotation specification portion and an input control. Therefore, the area range of the first component can be identified by image area identification as described above in step S104. Further, text of a second component within the first component may be identified by text recognition, and component attributes of the second component, e.g., text boxes, video boxes, etc., may be identified by component attribute recognition.

The first component may be identified at a first identification resolution; and identifying the second component at a second identification resolution, wherein the second identification resolution is greater than the first resolution. In this way, a smaller recognition resolution is used when recognizing the larger first component, thereby saving processing resources in recognition. While a higher recognition resolution may be employed when recognizing smaller second components, thereby increasing the accuracy of the recognition. Here, the smaller recognition resolution when the first component is recognized is relative to the resolution when the second component is recognized. The recognition resolution may be considered as the size of the image patch at the time of image recognition. The higher the recognition resolution is, the smaller the image blocks used for recognition are; conversely, the larger the image block. In this way, a balance can be found between processing efficiency and recognition accuracy.

Further, the manner in which the components are identified can be determined automatically. For example, a first property of a first component may be first determined, where the first property indicates that the first component is a text box, a video box, or a control. Then, a first identification mode for identifying the first component is determined based on the first property. As described above, the first recognition means may include at least one of image region recognition, text recognition, and component attribute recognition. Then, the first component is identified in a first identification manner.

The second component may be automatically identified in a similar manner. First, a second property of a second component is determined, wherein the second property indicates that the second component is a text box, a video box, or a control. Then, a second identification mode for identifying the second component is determined based on the second property, wherein the second identification mode comprises at least one of image region identification, text identification and component attribute identification. Then, the second component is identified in a second identification manner.

By identifying the property of the component, higher flexibility can be provided for the identification operation, and the way of identifying the design script is enriched. Further, subsequent code building work for the identified components may also be facilitated. For example, the control can be automatically set to an available or unavailable state, and the like.

The above-described processing can be realized by setting a configuration file. The configuration file indicates a correspondence between the property and the identification manner, that is, a correspondence between the first property or the second property and the first identification manner or the second identification manner. The same configuration file may be shared in the identification of the first component and the second component. The first recognition mode and the second recognition mode may be determined using a configuration file.

The user can set the profile as desired. In this way, configurability of the design draft recognition system and the like can be increased.

For example, a design may include a plurality of larger components, and a plurality of smaller components within each larger component. Wherein the plurality of first components are separate from each other.

In addition to identifying components in the design by range size, respectively, in a recursive manner, different information in the design may be identified using such a recursive manner. For example, first information in the first component and the second component may be identified in a first identification process. Next, in a second identification process, second information in the first component and the second component is identified. And recording the first information and the second information and the corresponding first component and second component in a linked list mode. The first information is, for example, text information in a design draft, and the second information is, for example, picture information in the design draft. In this way, the design may be recorded in a linear fashion, thereby simplifying storage and management of components in the design.

In step S112, a web page design provided by the user is acquired. The web page design is in picture format.

At step S114, at least one first component in the web page design is identified.

In step S116, a first component to perform further identification is selected.

In step S118, among the selected first components, at least one second component is identified.

In step S120, elements on the web page are automatically generated based on the identified first component and second component. Here, codes of different components may be preset, for example, a code of a text box, a code of a video box, a code of a determination button may be preset. The designer may make further modifications as needed, e.g., adding a network address of a video frame, etc., based on the automatically generated page code.

FIG. 3 shows a schematic block diagram of an apparatus to recognize a design according to one embodiment. The apparatus may be provided in the designer's computer. The apparatus may also be integrated into a common design platform.

As shown in fig. 3, apparatus 20 for recognizing a design includes an acquisition module 22, a first recognition module 24, a selection module 26, and a second recognition module 28.

The acquisition module 22 acquires a design draft provided by a user. The design is in picture format.

The first recognition module 24 recognizes at least one first component in the design.

The selection module 26 selects the first component for which further identification is to be performed.

The second identification module 28 identifies at least one second component among the selected first components.

The apparatus 20 for recognizing a design of fig. 3 may implement the scheme in the embodiment of fig. 1.

For example, first recognition module 24 may determine a first outline of the first component based on a color change in the design, and trim the first component from the design along the first outline. Second recognition module 28 may determine a second outline of the second component based on the color change in the first component and trim the first component from the design draft along the second outline.

The first identification module 24 may identify the first component at a first identification resolution. The second identification module 28 may identify the second component at a second identification resolution. The second recognition resolution is greater than the first resolution.

First identification module 24 may also determine a first property of the first component, wherein the first property indicates that the first component is a text box, a video box, or a control; determining a first identification mode for identifying the first component based on the first property, wherein the first identification mode comprises at least one of image region identification, text identification and component attribute identification; and identifying the first component in a first identification mode.

Second identification module 28 may also determine a second property of the second component, wherein the second property indicates that the second component is a text box, a video box, or a control; determining a second identification mode for identifying the second component based on the second property, wherein the second identification mode comprises at least one of image region identification, text identification and component attribute identification; and identifying the second component in a second identification manner.

The apparatus 20 may also include a configuration file. The configuration file comprises the corresponding relation between the first property or the second property and the first identification mode or the second identification mode. The first identification means and/or the second identification means may be determined using a configuration file.

In addition, the apparatus 20 may also include a recording module (not shown). The recording module may record a nesting relationship of the first component and the second component based on the identified order of the first component and the second component.

In addition to this, the present invention is,

in addition, processing may be performed for different information. For example, in the first recognition process, the first recognition module 24 and the second recognition module 28 recognize the first information in the first component and the second component; in the second recognition process, the first recognition module 24 and the second recognition module 28 recognize the second information in the first component and the second component. The recording module records the first information and the second information and the corresponding first component and second component in a linked list mode.

The present specification also provides an electronic device. Fig. 4 shows an electronic device 300.

The electronic device 300 may be used for page design and may be used to identify a design draft for page design.

As shown in fig. 4, the electronic device 300 includes a processor 302, a memory 304. Electronic device 300 may also include a display screen 310, a user interface 312, a camera 314, an audio/video interface 316, a sensor 318, and a communications component 320, among other things. In addition, the electronic device 300 may further include a power management chip 306, a battery 308, and the like. The electronic device 300 may be a smartphone, tablet computer, laptop computer, desktop computer, or the like.

The processor 302 may be various processors. For example, it may be an ARM architecture processor.

The memory 304 may store the underlying software, system software, application software, data, etc. needed for the electronic device 300 to operate. The memory 304 may include various forms of memory, such as ROM, RAM, Flash, etc.

The display screen 310 may be a liquid crystal display screen, an OLED display screen, or the like. In one example, the display screen 310 may be a touch screen. The user can perform an input operation through the display screen 210. In addition, the user can also perform fingerprint identification and the like through the touch screen.

The user interface 312 may include a USB interface, a lightning interface, a keyboard, and the like.

The camera 314 may be a single camera or multiple cameras. In addition, the camera 314 may be used for face recognition by the user.

The audio/video interface 316 may include, for example, a speaker interface, a microphone interface, a video transmission interface such as HDMI, and the like.

The sensors 318 may include, for example, gyroscopes, accelerometers, temperature sensors, humidity sensors, pressure sensors, and the like. For example, the environment around the electronic device may be determined by sensors, etc.

The communication component 320 may include, for example, a WiFi communication component, a bluetooth communication component, a 3G, 4G, and 5G communication component, and the like. Through the communication section 320, the electronic apparatus 300 can be arranged in a network.

The power management chip 306 can be used to manage the power input to the electronic device 300 and also manage the battery 308 to ensure greater utilization efficiency. The battery 308 is, for example, a lithium ion battery or the like.

The electronic device shown in fig. 4 is merely illustrative and is in no way intended to limit the present invention, its application, or uses.

The memory 304 of the electronic device 300 may store executable instructions. The executable instructions, when executed by the processor 302, implement the above method of identifying a design.

Here, a storage medium may also be provided that stores executable instructions that, when executed, implement the method illustrated in fig. 1 or fig. 2 above.

Fig. 5-8 illustrate an example of a design according to one recognition.

A design 40 for an electronic device such as a smartphone is schematically shown in fig. 5.

The designer may manually draw the design and then input the drawn design into a computer for processing using a scanning device. In addition, the designer may also use drawing software to complete the design and save the design in the computer.

As shown in fig. 5,

larger assemblies

50 and 60 are included in design 40. Included in the assembly 50 are a video box 51, a text box 52, and a button control 53. Included in the assembly 60 are a menu 61, a picture 62 and controls 63.

As shown in fig. 6, the component 50 is first identified. Next, the components of the components 50, i.e., the video box 51, the text box 52, and the button controls 53, are identified. Because the component 50 is cropped, the components 51-53 can be identified with a higher resolution and different identification methods can be used for different components. For example, the meaning of the annotation "date" of the component 52 may be identified. In the event that "date" is identified, the input format in the text box of the component 52 may be identified as "DD-MM-YYYY". By the method, the automation degree of the system is improved, and the processing amount of subsequent codes is reduced.

As shown in fig. 7, the component 60 is first identified. Next, the smaller of the components 60, namely, the menu 61, the picture 62, and the controls 63 are identified. For example, when the determination component 63 is a control, it can be identified whether the control 63 is available. In fig. 6, for example, the control 63 is in an unavailable state.

The identification process may be performed based on a configuration file.

As shown in fig. 8, by the process of record recognition, the nested structure of the design draft 40 can be naturally obtained. Here, the design 40 includes

components

50, 60; assembly 50 includes assemblies 51-53; and assembly 60 includes assemblies 61-63.

Further, it is also possible to input the entire design into a computer and then perform the above-described processing for different information. For example, in the first recognition process, text information is extracted, whereby the text information in the

components

52, 53, 61 can be recognized; in the first recognition process, picture information is extracted, e.g., a picture of the component 62 can be obtained, and so on.

Software planners, front-end page rendering personnel, and software module designers can work in concert to implement the design of the front-end page. Fig. 9 and 10 show different application scenarios.

As shown in FIG. 9, software planner A, front end page mapper B, and software module designer C may connect to network 710 via

computers

742, 744, 746, respectively.

Servers

722, 724, etc. may be provided in the network 710 for storing files uploaded by software planner a, front end page mapper B, and software module designer C, and for storing collaborative software for collaborative work. Personnel A, B, C may accomplish front-end page design tasks through collaborative software.

For example, software planner A first proposes a requirement for the front end page design and uses computer 742 to send the requirement over network 710 to front end page renderer B's computer 744. This requirement can be communicated from computer 742 to computer 744 in a number of ways. For example, software planner A may use computer 742 to send documents documenting the requirements to front end page mapper B's computer 744 via mail or instant messaging software. In this case, the

servers

722, 724 may be mail servers or instant messaging servers. Further, the above-described operations may also be implemented using cooperative software. For example, a main program of the cooperative software is disposed in the

servers

722, 724, and a terminal program of the cooperative software is disposed in the

computers

742, 744, 746. When the software planner A uploads the requirements to the main program using the terminal program in computer 742, the main program automatically distributes the task of drawing the design draft to the front-end page drafter B via computer 744.

Next, the front-end page renderer B renders the design draft. The front-end page drafter B may draft the design in a variety of ways. For example, as previously described, the front end page renderer B may manually render the design with a brush and input the design into the computer 744 using a scanner. Front end page rendering personnel B may also use the drawing software in computer 744 to render the design. After the front-end page renderer B has finished rendering the design draft, he sends the design draft to the

servers

722, 724 or software module designer C in the network using the computer 744. The components in the design may be automatically identified by development software in the

servers

722, 724 or the computer 746 according to the schemes in the previous embodiments and a preliminary front end page code generated using the code template.

Similar to the manner of delivery described above, the design or file containing the preliminary front end page code may also be delivered to the computer 746 in a variety of manners. Similar descriptions will not be repeated here.

Next, the software module designer C supplements the preliminary front end page code, e.g., adding functionality for controls, adding video addresses in video frames, etc. And after the software module designer C completes all component codes, completing the development task of the front-end page and generating a corresponding development file. Software module designer C may upload the completed development file to

servers

722, 724 for external release.

The network shown in fig. 9 may be an internal network for software development or a public network. Software planner a can publish the required tasks in the internal network, draw design drafts by internal front-end page drafters B, and complete code development by internal software module designers C. Optionally, the software planner a may also publish the required task to a public network platform, and the front-end page renderer B obtains the required task and completes the design draft. Next, the software module designer C acquires the design draft through the public network platform and adds the code. Finally, software module designer C returns the completed file to software planner a. Software planner a publishes the designed front-end page onto the network as needed.

Fig. 10 shows an example of the draft collaborative design system. The co-design system may be deployed in a network environment as shown in fig. 9.

As shown in fig. 10, the design draft collaborative design system includes a main program 80 and a terminal program 81. The main program 80 is also connected to a file storage system 82 and development software 83. The terminal program 81 may be arranged in the

computers

742, 744, 746 shown in fig. 9. The main program 80, file storage system 82, and development software 83 may be deployed in

servers

742, 744.

For example, the terminal programs 81 are arranged in the computers of the software planner a, the front-end page renderer B, and the software module designer C, respectively. Software planner A first completes the requirements for the front-end page design. Software planner a then clicks complete in the terminal program. The demand is then uploaded to main program 80. The main program 80 may store the requirement in the file storage system 82 and automatically distribute the task containing the requirement to the terminal program of the front end page-renderer B. As described above, the front-end page rendering worker B starts rendering the design draft after receiving the required task. After the front-end page renderer B completes rendering of the design, he clicks completed in the terminal program, and the design is uploaded to the main program 80. The main program 80 stores the received design draft in the file storage system 82. The primary program 80 may send the design into the development software 83 to identify the development and form a preliminary front end page development file. Then, main program 80 sends the preliminary front end page development file to the terminal program of software module designer C to further add code. Alternatively, the recognition processing of the design draft may be performed in a terminal program of the software module designer C. And finally, completing the code adding task of the front-end page by the software module designer C and clicking to complete the code adding task. The terminal program of software module designer C sends the completed code to main program 80. Main program 80 stores these codes in file storage system 82 and issues the codes as needed to complete the development task for the front-end page.

The embodiments in the present specification are described in a progressive manner, and the same and similar parts among the embodiments are referred to each other, and each embodiment focuses on the differences from the other embodiments. In particular, for system embodiments, since they are substantially similar to method embodiments, the description is relatively simple, and for related matters, reference may be made to some descriptions of method embodiments

The foregoing description has been directed to specific embodiments of this disclosure. Other embodiments are within the scope of the following claims. In some cases, the actions or steps recited in the claims may be performed in a different order than in the embodiments and still achieve desirable results. In addition, the processes depicted in the accompanying figures do not necessarily require the particular order shown, or sequential order, to achieve desirable results. In some embodiments, multitasking and parallel processing may also be possible or may be advantageous.

Claims

1. A method of identifying a design, comprising:

acquiring a design draft provided by a user, wherein the design draft is in a picture format;

identifying at least one first component in the design;

selecting a first component for which further identification is to be performed; and

among the selected first components, at least one second component is identified.

2. The method of claim 1, further comprising:

in a first recognition process, recognizing first information in the first and second components;

in a second identification process, identifying second information in the first and second components; and

and recording the first information and the second information and the corresponding first component and second component in a linked list mode.

3. The method of claim 1, wherein identifying at least one first component in the design comprises:

determining a first outline of the first component based on the color change in the design draft; and

cutting the first assembly from the design draft along the first contour,

wherein identifying at least one second component comprises:

determining a second contour of the second component based on the color change in the first component; and

and cutting the first assembly from the design draft along the second contour.

4. The method of claim 1, wherein identifying at least one first component in the design comprises: identifying a first component at a first identification resolution; and

identifying at least one second component comprises: identifying a second component at a second identification resolution,

wherein the second recognition resolution is greater than the first resolution.

5. The method of claim 1, wherein identifying at least one first component in the design comprises:

determining a first property of a first component, wherein the first property indicates that the first component is a text box, a video box, or a control;

determining a first identification mode for identifying the first component based on the first property, wherein the first identification mode comprises at least one of image region identification, text identification and component attribute identification; and

the first component is identified in a first identification manner.

6. The method of claim 5, wherein identifying at least one second component comprises:

determining a second property of a second component, wherein the second property indicates that the second component is a text box, a video box, or a control;

determining a second identification mode for identifying the second component based on the second property, wherein the second identification mode comprises at least one of image region identification, text identification and component attribute identification; and

the second component is identified in a second identification manner.

7. The method of claim 6, further comprising:

setting a configuration file, wherein the configuration file comprises the corresponding relation between the first property or the second property and the first identification mode or the second identification mode,

wherein determining a first identification manner for identifying the first component comprises: using the configuration file, determining a first identification means,

wherein determining a second identification manner for identifying the second component comprises: using the configuration file, a second identification mode is determined.

8. The method of claim 1, further comprising:

and recording the nesting relation of the first component and the second component based on the identification sequence of the first component and the second component.

9. A method of automatically generating a web page, comprising:

acquiring a webpage design draft provided by a user, wherein the webpage design draft is in a picture format;

identifying at least one first component in the webpage design;

selecting a first component for which further identification is to be performed;

identifying, among the selected first components, at least one second component; and

based on the identified first component and second component, elements on the web page are automatically generated.

10. An apparatus for recognizing a design, comprising:

the acquisition module acquires a design draft provided by a user, wherein the design draft is in a picture format;

the first identification module identifies at least one first component in the design draft;

a selection module that selects a first component for which further identification is to be performed; and

a second identification module that identifies at least one second component among the selected first components.

11. An electronic device comprising a processor and a memory, the memory storing executable instructions that, when the electronic device is operated, control the processor to perform the method according to any of claims 1-9.

12. A storage medium storing executable instructions that, when executed, implement a method according to any one of claims 1-9.