WO2022145723A1 - Procédé et appareil de détection de disposition - Google Patents

Procédé et appareil de détection de disposition Download PDF

Info

Publication number
WO2022145723A1
WO2022145723A1 PCT/KR2021/016801 KR2021016801W WO2022145723A1 WO 2022145723 A1 WO2022145723 A1 WO 2022145723A1 KR 2021016801 W KR2021016801 W KR 2021016801W WO 2022145723 A1 WO2022145723 A1 WO 2022145723A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
component
current image
bounding box
box information
Prior art date
Application number
PCT/KR2021/016801
Other languages
English (en)
Inventor
Xinyan Mei
Juan Liu
Dohun Kim
Longhai WU
Jie Chen
Original Assignee
Samsung Electronics Co., Ltd.
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Samsung Electronics Co., Ltd. filed Critical Samsung Electronics Co., Ltd.
Publication of WO2022145723A1 publication Critical patent/WO2022145723A1/fr

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
    • G06V10/774—Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/22—Matching criteria, e.g. proximity measures
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]

Definitions

  • Embodiments of the present disclosure relate to the field of computer technology, and specifically to a method and apparatus for detecting a layout.
  • AI applies human skills and tendencies to inanimate objects, and thus begins to affect the automation test of the inanimate objects in various ways.
  • the reinforcement learning models needs to record states corresponding to each action, i.e., current page image information.
  • the layout of the current image information is relatively stable, but the content thereof is constantly refreshed as a third-party app is updated over time. Therefore, in order to obtain the similarity and difference between states of different images, it is required to extract the layout information of the images.
  • the conventional image segmentation method such as faster-rcnn and MASK-RCNN, each local portion of an image may be segmented, but the global information (e.g., a layout alignment) between local portions is ignored.
  • Embodiments of the present disclosure propose a method and apparatus for detecting a layout, an electronic device and a computer readable medium.
  • the embodiments of the present disclosure provide a method for detecting a layout.
  • the method comprises: recognizing a current image including at least one component in response to acquiring the current image, and obtaining bounding box information and a category name of each component by using an instance segmentation model; generating a marked image corresponding to the current image based on the bounding box information of the each component; and inputting the bounding box information and the category name of the each component and the marked image into a long short-term memory network detection model and outputting a layout result of the component in the current image, the layout result including sorting and grouping results of the each component in the current image.
  • the inputting the bounding box information and the category name of the each component and the marked image into a long short-term memory network detection model and outputting a layout result of the component in the current image comprises: inputting the bounding box information and the category name of the each component into an encoding model in the long short-term memory network detection model, and outputting text feature information corresponding to the each component; inputting the marked image into an image processing model in the long short-term memory network detection model, and outputting image feature information corresponding to the marked image; and inputting the text feature information and the image feature information into a decoding model in the long short-term memory network detection model, and outputting the layout result of the component in the current image.
  • the instance segmentation model is acquired by: acquiring a sample image set, wherein the sample image set includes at least one sample image, and each sample image includes at least one component; ascertaining and annotating a category name and bounding box information of each component in the each sample image; training and obtaining the instance segmentation model by using the each sample image as an input, and by using the category name and the bounding box information of the each component in the each sample image as a desired output.
  • the method further comprises: performing a state similarity search on stored images based on the layout result of the component in the current image, to obtain a plurality of similar images; and comparing the current image with each similar image to determine whether the current image and the each similar image are in a given state.
  • the comparing comprises: calculating an intersection-over-union between the layout result of the component in the current image and a layout result of a component in the each similar image; comparing a pixel of the current image with a pixel of the each similar image to obtain a comparison result between the pixel of the current image and the pixel of the each similar image; and comparing the intersection-over-union and the comparison result with a preset(or predetermined) threshold value to determine whether the current image and the each similar image are in the given state.
  • the method further comprises: deleting the current image in response to ascertaining that the current image and the similar image are in the given state; and storing the current image in response to ascertaining that the current image and the similar image are in different states.
  • the method further comprises: performing, for each component having a given layout, an adjustment operation on bounding box information of the each component to obtain adjusted bounding box information corresponding to the each component, in response to acquiring the layout result of the component in the current image.
  • the method further comprises: obtaining a new sample image set by using the current image, the category name of the each component and the adjusted bounding box information; and training the instance segmentation model based on the new sample image set.
  • the method further comprises: inputting the adjusted bounding box information corresponding to the each component and the layout result into a UI2Code to obtain a duplicate layout code, in response to acquiring the adjusted bounding box information corresponding to the each component.
  • the embodiments of the present disclosure provide an apparatus for detecting a layout.
  • the apparatus comprises: a recognizing module, configured to recognize a current image including at least one component in response to acquiring the current image, and obtain bounding box information and a category name of each component, by using an instance segmentation model; a generating module, configured to generate a marked image corresponding to the current image based on the bounding box information of the each component; and an outputting module, configured to input the bounding box information and the category name of the each component and the marked image into a long short-term memory network detection model and output a layout result of the component in the current image, the layout result including sorting and grouping results of the each component in the current image.
  • the outputting module comprises: an encoding unit, configured to input the bounding box information and the category name of the each component into an encoding model in the long short-term memory network detection model, and output text feature information corresponding to the each component; an image processing unit, configured to input the marked image into an image processing model in the long short-term memory network detection model, and output image feature information corresponding to the marked image; and a decoding unit, configured to input the text feature information and the image feature information into a decoding model in the long short-term memory network detection model, and output the layout result of the component in the current image.
  • the instance segmentation model is acquired by: acquiring a sample image set, wherein the sample image set includes at least one sample image, and each sample image includes at least one component; ascertaining and annotating a category name and bounding box information of each component in the each sample image; training and obtaining the instance segmentation model by using the each sample image as an input, and by using the category name and the bounding box information of the each component in the each sample image as a desired output.
  • the apparatus further comprises: a searching module, configured to perform a state similarity search on stored images based on the layout result of the component in the current image, to obtain a plurality of similar images; and a determining module, configured to compare the current image with each similar image to determine whether the current image and the each similar image are in a given state.
  • a searching module configured to perform a state similarity search on stored images based on the layout result of the component in the current image, to obtain a plurality of similar images
  • a determining module configured to compare the current image with each similar image to determine whether the current image and the each similar image are in a given state.
  • the determining module comprises: a calculating unit, configured to calculate an intersection-over-union between the layout result of the component in the current image and a layout result of a component in the each similar image; a comparing unit, configured to compare a pixel of the current image with a pixel of the each similar image to obtain a comparison result between the pixel of the current image and the pixel of the each similar image; and a determining unit, configured to compare the intersection-over-union and the comparison result with a preset threshold value to determine whether the current image and the each similar image are in the given state.
  • the apparatus further comprises: a deleting module, configured to delete the current image in response to ascertaining that the current image and the similar image are in the given state; and a storing module, configured to store the current image in response to ascertaining that the current image and the similar image are in different states.
  • the apparatus further comprises: an adjusting module, configured to perform, for each component having a given layout, an adjustment operation on bounding box information of the each component to obtain adjusted bounding box information corresponding to the each component, in response to acquiring the layout result of the component in the current image.
  • an adjusting module configured to perform, for each component having a given layout, an adjustment operation on bounding box information of the each component to obtain adjusted bounding box information corresponding to the each component, in response to acquiring the layout result of the component in the current image.
  • the apparatus further comprises: an updating module, configured to obtain a new sample image set by using the current image, the category name of the each component and the adjusted bounding box information; and a training module, configured to train the instance segmentation model based on the new sample image set.
  • the apparatus further comprises: a code generating module, configured to input the adjusted bounding box information corresponding to the each component and the layout result into a UI2Code to obtain a duplicate layout code, in response to acquiring the adjusted bounding box information corresponding to the each component.
  • a code generating module configured to input the adjusted bounding box information corresponding to the each component and the layout result into a UI2Code to obtain a duplicate layout code, in response to acquiring the adjusted bounding box information corresponding to the each component.
  • the present disclosure provides an electronic device, the electronic device comprising one or more processors and a storage apparatus configured to store one or more programs.
  • the one or more programs when executed by the one or more processors, cause the one or more processors to implement the method described in any implementation in the first aspect.
  • the present disclosure provides a computer readable medium storing a computer program.
  • the program when executed by a processor, implements the method described in any implementation in the first aspect.
  • the current image including the at least one component is recognized using the instance segmentation model in response to acquiring the current image, to obtain the bounding box information and the category name of the each component. Then, the marked image corresponding to the current image is generated based on the bounding box information of the each component. Finally, the bounding box information and the category name of the each component and the marked image are inputted into the long short-term memory network detection model, and the layout result of the component in the current image is outputted.
  • the layout result includes the sorting and grouping results of the each component in the current image, thus implementing that the layout result of the each component in the image is extracted.
  • the grouping result of the each component can be obtained based on the interdependence relationship between components. Accordingly, the global information between the portions of the current image is taken into consideration, thereby improving the accuracy of the layout result.
  • Fig. 1 is a diagram of an exemplary system architecture in which some embodiments of the present disclosure may be applied;
  • Fig. 2 is a flowchart of an embodiment of a method for detecting a layout according to the present disclosure
  • Fig. 3 is a schematic diagram of an application scenario of the method for detecting a layout according to the present disclosure
  • Fig. 4 is a schematic diagram of an embodiment of an output of a layout result according to the present disclosure.
  • Fig. 5 is a schematic diagram of another embodiment of the method for detecting a layout according to the present disclosure.
  • Fig. 6 is a schematic diagram of another embodiment of the method for detecting a layout according to the present disclosure.
  • Fig. 7 is a schematic structural diagram of an embodiment of an apparatus for detecting a layout according to the present disclosure.
  • Fig. 8 is a schematic structural diagram of an electronic device adapted to implement embodiments of the present disclosure.
  • Fig. 1 illustrates an exemplary system architecture 100 in which a method for detecting a layout or an apparatus for detecting a layout according to an embodiment of the present disclosure may be applied.
  • the system architecture 100 may include terminal devices 104 and 105, a network 106, and servers 101, 102 and 103.
  • the network 106 serves as a medium providing a communication link between the terminal devices 104 and 105 and the servers 101, 102 and 103.
  • the network 106 may include various types of connections, for example, wired or wireless communication links, or optical fiber cables.
  • a user may use the terminal devices 104 and 105 to interact with the servers 101, 102 and 103 via the network 106 to receive or send messages.
  • Various applications e.g., an image processing application, a data processing application, an instant communication tool, social platform software, a search application, and a shopping application
  • an image processing application e.g., an image processing application, a data processing application, an instant communication tool, social platform software, a search application, and a shopping application
  • the terminal devices 104 and 105 may be hardware or software.
  • the terminal devices may be various electronic devices having a display screen and supporting communication with the servers, the electronic devices including, but not limited to, a smartphone, a tablet computer, a laptop portable computer, a desktop computer, etc.
  • the terminal devices may be installed in the above listed electronic devices.
  • the terminal devices may be implemented as a plurality of pieces of software or a plurality of software modules, or as a single piece of software or a single software module, which will not be specifically defined here.
  • the terminal devices 104 and 105 may acquire a current image via the network 106 or through local reading.
  • the current image may include at least one component.
  • the terminal devices 104 and 105 call an instance segmentation model to recognize the current image to obtain bounding box information and a category name of each component in the current image.
  • the terminal devices 104 and 105 mark the current image according to the obtained bounding box information of the each component, thus obtaining a marked image including the bounding box information of the each component.
  • the terminal devices 104 and 105 input the bounding box information and the category name of the each component and the marked image into a long short-term memory network detection model.
  • the long short-term memory network detection model processes the inputted content, and outputs a layout result of a component in the current image.
  • the layout result may include a grouping result of the category name of the each component in the current image. That is, components in the same layout are considered as being in the same grouping.
  • the servers 101, 102 and 103 may be servers providing various services, for example, backend servers receiving a request sent by a terminal device communicated with the servers 101, 102 and 103.
  • the backend servers may perform processing such as receiving and analyzing on the request sent by the terminal device, and generate a processing result.
  • the servers may be hardware or software.
  • the servers may be various electronic devices providing various services for the terminal devices.
  • the servers may be implemented as a plurality of pieces of software or a plurality of software modules providing various services for the terminal devices, or as a single piece of software or a single software module providing various services for the terminal devices, which will not be specifically defined here.
  • the method for detecting a layout provided by the embodiments of the present disclosure may be performed by the terminal devices 104 and 105.
  • the apparatus for detecting a layout may be provided in the terminal devices 104 and 105.
  • terminal devices the numbers of the terminal devices, the networks, and the servers in Fig. 1 are merely illustrative. Any number of terminal devices, networks, and servers may be provided based on actual requirements.
  • Fig. 2 illustrates a flow 200 of an embodiment of a method for detecting a layout according to the present disclosure.
  • the method for detecting a layout may include the following steps.
  • Step 210 using an instance segmentation model to recognize a current image including at least one component in response to acquiring the current image, to obtain bounding box information and a category name of each component.
  • an executing body e.g., the terminal devices shown in Fig. 1
  • the method for detecting a layout may acquire the current image from a server via a network, acquire the current image by performing an operation such as a screen capture operation on a currently displayed image, or acquire the current image from a local storage.
  • the current image may include the at least one component, and a component refers to a simple encapsulation for some portion of data and some portion of a method in the image.
  • the current image may consist of a plurality of different components, and each component has a different attribute and plays a different role.
  • the above executing body may call the instance segmentation model to process the current image, and recognize the each component in the current image, to obtain the bounding box information and the category name of the each component.
  • the category names of components in the current image may be outputted as a sequence. That is, a category name sequence of the components in the current image is outputted.
  • the bounding box information of the component is used to represent a size of the component that is displayed using a border box, and generally includes the coordinates of each border box.
  • the category name of the component is used to represent the type name of the component, and may include a customized type name, for example, menu, scroll view, tool bar, user info(information), tip box, noti(notification) box, keyboard, text list, text box, bgimg(for example, background image), banner, other, view box, button, pop dialog, pop menu, Item, first screen and progress bar.
  • a customized type name for example, menu, scroll view, tool bar, user info(information), tip box, noti(notification) box, keyboard, text list, text box, bgimg(for example, background image), banner, other, view box, button, pop dialog, pop menu, Item, first screen and progress bar.
  • the above instance segmentation model may be used to segment each instance in an image to obtain a border box and a type of the each instance, for example, a Mask R-CNN model.
  • the Mask R-CNN model trained and obtained based on the above image and the bounding box information and the category name of each component in the image may perform a corresponding pre-processing operation on the inputted current image to obtain a pre-processed image, and then input the pre-processed image into a pre-trained neural network to obtain a corresponding feature map.
  • the Mask R-CNN model sets a predetermined number of points of interest for each point in the feature map to obtain a plurality of candidate points of interest, and puts the candidate points of interest into an RPN network for a binary classification and a BB regression, to filter out a portion of the candidate points of interest.
  • the Mask R-CNN model performs an ROIAlign(Region Of Interest Align) operation on the remaining points of interest (i.e., first makes a pixel of an original image corresponding to a pixel of the feature map, and then makes the feature map corresponding to a fixed feature), and finally performs a classification (N category classification), a BB regression and MASK generation (performs an FCN operation in each point of interest) on these candidate points of interest, thus obtaining the bounding box information and the category name of the each component in the current image.
  • ROIAlign(Region Of Interest Align) operation i.e., first makes a pixel of an original image corresponding to a pixel of the feature map, and then makes the feature map corresponding to a fixed feature
  • a classification N category classification
  • BB regression and MASK generation performs an FCN operation in each point of interest
  • Step 220 generating a marked image corresponding to the current image based on the bounding box information of the each component.
  • the above executing body replaces the each component in the current image with a bounding box according to the bounding box information of the each component, that is, changes the each component in the current image to the corresponding bounding box to obtain a marked image including each bounding box.
  • the above bounding box information is displayed using the border box.
  • the above executing body After obtaining a rectangular box of the each component in the current image, the above executing body replaces the each component with the corresponding rectangular box, and uses an image including the rectangular box as the marked image corresponding to the current image.
  • Step 230 inputting the bounding box information and the category name of the each component and the marked image into a long short-term memory network detection model, and outputting a layout result of the component in the current image.
  • the above executing body inputs the bounding box information and the category name of the each component and the marked image into the long short-term memory network detection model.
  • the long short-term memory network detection model processes the inputted content, and outputs the layout result of the component in the current image.
  • the layout result may include sorting and grouping results of the each component in the current image. Since there is an association dependency relationship between the components in the current image, the long short-term memory network detection model may sort and group the components in the current image according to the association dependency relationship between the components, thereby obtaining the sorting and grouping results of the each component in the current image.
  • the above executing body may sort and group the category name of a component in the above category name sequence, to obtain sorting and grouping results of the category name corresponding to the component.
  • Category names in the same grouping may be distinguished in the same way, for example, the category names in the same grouping are put into the same brackets. Different groupings are distinguished using different brackets.
  • Components in the same grouping belong to the same layout distribution, and there may be different alignments such as a left alignment, a vertical middle alignment, a right alignment, a top alignment, a horizontal center alignment, and a bottom alignment.
  • Fig. 3 is a schematic diagram of an application scenario of the method for detecting a layout according to this embodiment.
  • the method may be applied to the application scenario of Fig. 3.
  • a television 301 captures a current image displayed on the screen to obtain a current image.
  • the television 301 recognizes the current image using an instance segmentation model, to obtain bounding box information and a category name of each component in the current image.
  • the television 301 generates a marked image corresponding to the current image according to the bounding box information of the each component in the current image, and inputs the bounding box information and the category name of the each component and the marked image into a long short-term memory network detection model, to obtain an outputted layout result 302 of a component in the current image.
  • the television 301 acquires a component list identical and similar to the layout result 302 of the current image from a local focus manager according to the obtained layout result 302.
  • the television 301 analyzes the received voice to ascertain whether the voice includes a similar command such as "Next" and "Previous.”
  • the voice includes the similar command such as "Next” and "Previous”
  • the television 301 first performs a selection in the component list identical to the layout result 302 of the current image, and presents the selected image to the user through the screen. If the selection for the image in the component list identical to the layout result 302 of the current image is completed, the component list is switched to the similar list to perform a selection.
  • the current image including the at least one component is recognized using the instance segmentation model in response to acquiring the current image, to obtain the bounding box information and the category name of the each component. Then, the marked image corresponding to the current image is generated based on the bounding box information of the each component. Finally, the bounding box information and the category name of the each component and the marked image are inputted into the long short-term memory network detection model, and the layout result of the component in the current image is outputted.
  • the layout result of the each component in the image is extracted.
  • the grouping result of the each component can be obtained based on the interdependence relationship between the components. Accordingly, the global information between the portions of the current image is taken into consideration, thereby improving the accuracy of the layout result.
  • Fig. 4 illustrates the step 230 "inputting the bounding box information and the category name of the each component and the marked image into a long short-term memory network detection model, and outputting a layout result of the component in the current image" in Fig. 2, which may be implemented based on the following steps:
  • Step 410 inputting bounding box information and a category name of each component into an encoding model in a long short-term memory network detection model, and outputting text feature information corresponding to the each component.
  • the above executing body first applies embedded coding to the bounding box information and the category name of the each component to obtain vector information of a bounding box and a category, then inputs the obtained vector information of the bounding box and the category into the encoding model in the long short-term memory network detection model (LSTM), and outputs the text feature information corresponding to the each component.
  • LSTM long short-term memory network detection model
  • Step 420 inputting a marked image into an image processing model in the long short-term memory network detection model, and outputting image feature information corresponding to the marked image.
  • the above executing body inputs the marked image into the image processing model in the long short-term memory network detection model to perform an operation such as a convolution operation and a full connection operation of a CNN, and outputs the image feature information corresponding to the marked image.
  • Step 430 inputting the text feature information and the image feature information into a decoding model in the long short-term memory network detection model, and outputting a layout result of a component in a current image.
  • the above executing body fuses the text feature information and the image feature information to obtain the fused feature information. Then, the fused feature information is inputted into the decoding model in the long short-term memory network detection model, and a softmax classification is performed. A layout result having the same length as an inputted category name sequence is outputted, that is, the layout result of the component in the current image is outputted.
  • the layout result is detected based on a long short-term memory network, and thus, the problem that features are dependent on each other may be solved.
  • a grouping result of the each component is ascertained based on the interdependence relationship between the features, and thus, a relationship between local portions is taken into consideration, thereby improving the accuracy of the grouping result.
  • the above instance segmentation model is acquired based on the following steps:
  • a sample image set is acquired.
  • the process of training the above instance segmentation model is implemented in a server, and the server may locally read the sample image set or acquire the sample image set from the above executing body.
  • the sample image set includes at least one sample image, and each sample image includes at least one component.
  • a category name and bounding box information of each component in the each sample image are ascertained and annotated.
  • the server may annotate the category name of the each component in the each sample image according to a customized category name. Moreover, the server annotates a border box to which the each component in the each sample image belongs, to implement the annotation for the bounding box information of the each component. Thus, the category name and the bounding box information that correspond to the each component in the each sample image are obtained.
  • the each sample image is used as an input, and the category name and the bounding box information of the each component in the each sample image are used as a desired output, to train and obtain the instance segmentation model.
  • the server may acquire a Mask R-CNN network, use a sample image as an input, and use a category name and bounding box information of each component corresponding to the inputted sample image as a desired output to train the Mask R-CNN network to obtain the trained Mask R-CNN network as the instance segmentation model.
  • the trained instance segmentation model may recognize an inputted image and output a category name and bounding box information of each component in the image.
  • the accuracy and the recognition efficiency of the category name and the bounding box information of the each component in the image may be improved.
  • Fig. 5 illustrates a flow 500 of another embodiment of the method for detecting a layout.
  • the method for detecting a layout may further include the following steps.
  • Step 510 using an instance segmentation model to recognize a current image including at least one component in response to acquiring the current image, to obtain bounding box information and a category name of each component.
  • step 510 is the same as step 210 in the embodiment shown in Fig. 2, which will not be repeatedly described here.
  • Step 520 generating a marked image corresponding to the current image based on the bounding box information of the each component.
  • step 520 is the same as step 220 in the embodiment shown in Fig. 2, which will not be repeatedly described here.
  • Step 530 inputting the bounding box information and the category name of the each component and the marked image into a long short-term memory network detection model, and outputting a layout result of the component in the current image.
  • step 530 is the same as step 230 in the embodiment shown in Fig. 2, which will not be repeatedly described here.
  • Step 540 performing a state similarity search on stored images based on the layout result of the component in the current image, to obtain a plurality of similar images.
  • the above executing body performs a state search on an image stored in a local database, that is, performs the state similarity search on the stored images according to the layout result of the component in the current image, to search an image in the stored images that is similar to the grouping result of the component in the current image or search an image having a similar layout in the stored images, thereby obtaining a plurality of similar images having a state similar to that of the layout result of the component in the current image.
  • Step 550 comparing the current image with each similar image to determine whether the current image and the each similar image are in a given state.
  • the above executing body compares the current image with the each similar image, respectively, and thus, a plurality of comparison results between the current image and the each similar image may be obtained.
  • a comparison result may include a similarity numerical value and the like. Then, the above executing body determines whether the current image and the each similar image are in the given state according to the comparison result.
  • the above executing body acquires 3 similar images that are respectively an image 1, an image 2, and an image 3.
  • the above executing body compares the current image with the image 1 to obtain a first similarity numerical value, then compares the current image with the image 2 to obtain a second similarity numerical value, and then compares the current image with the image 3 to obtain a third similarity numerical value.
  • the above executing body determines whether the current image and the each similar image are in the given state according to the first similarity numerical value, the second similarity numerical value and the third similarity numerical value.
  • the search for the image is implemented through the layout result of the component in the current image, thus implementing the state comparison between images, and the accuracy of the state comparison between the images can be improved based on the layout result of the component in the current image.
  • step 550 comparing the current image with each similar image to determine whether the current image and the each similar image are in a given state
  • the above step 550 may be implemented based on the following steps:
  • an intersection-over-union between the layout result of the component in the current image and a layout result of a component in the each similar image is calculated.
  • the above executing body acquires the layout result of the each similar image. Then, the above executing body ascertains the bounding box information of the each component in the current image and bounding box information of the component in the each similar image, and calculates an intersection-over-union between the bounding box information of the component in the current image and the bounding box information of the component in the each similar image, respectively.
  • the intersection-over-union (IOU) is a concept used in a target detection, which refers to an overlap ratio between a generated candidate bound and a ground truth bound, that is, a ratio between an intersection and a union of the candidate bound and the ground truth bound.
  • the most ideal situation refers to that the intersection and the union are completely overlapped, that is, the ratio is 1.
  • the above executing body may calculate an intersection-over-union between a border box of the component in the current image and a border box of a component in a similar image, to obtain the intersection-over-union between the layout result of the component in the current image and the layout result of the component in the each similar image, respectively.
  • a pixel of the current image and a pixel of the each similar image are compared to obtain a comparison result between the pixel of the current image and the pixel of the each similar image.
  • the above executing body acquires the pixel of the each similar image. Then, the above executing body compares the pixel of the current image with the pixel of the each similar image, respectively. The above executing body may calculate a difference value between the pixel of the current image and the pixel of the each similar image, and use the difference value as the comparison result between the pixel of the current image and the pixel of the each similar image.
  • intersection-over-union and the comparison result are compared with a preset threshold value to determine whether the current image and the each similar image are in the given state.
  • the above executing body compares the intersection-over-union and the comparison result that correspond to the each similar image with the preset threshold value, respectively.
  • the above executing body may add the intersection-over-union and the comparison result that correspond to the each similar image together, and compare the result of the addition with a preset threshold value, to determine whether the current image and the each similar image are in the given state.
  • the above executing body may determine whether the result of the addition is not less than the preset threshold value. If the result of the addition is not less than the preset threshold value, it is ascertained that the similar image corresponding to the result of the addition is in the same state as that of the current image. If the result of the addition is less than the preset threshold value, it is ascertained that the similar image corresponding to the result of the addition is in a state different from that of the current image.
  • the above executing body may also calculate the average value of the intersection-over-union and the comparison result that correspond to the each similar image, respectively, and compare the average value with a preset threshold value, to determine whether the current image and the each similar image are in the given state.
  • the above executing body may determine whether the average value is not less than the preset threshold value. If the result of the addition (average value) is not less than the preset threshold value, it is ascertained that the similar image corresponding to the average value is in the same state as that of the current image. If the average value is less than the preset threshold value, it is ascertained that the similar image corresponding to the average value is in a state different from that of the current image.
  • the intersection-over-union between the current image and the similar image and the comparison result between the pixels are calculated to determine whether the current image and the similar image are in the given state.
  • the accuracy of the state comparison is improved based on the plurality of determinations.
  • the method for detecting a layout may further include the following steps.
  • Step 560 deleting the current image in response to ascertaining that the current image and the similar image are in the given state.
  • the above executing body ascertains that the current image and the similar image are in the given state by comparing the intersection-over-union and the comparison result with the preset threshold value. It indicates that an image having the layout result of the current image is already stored in the local database, and thus, the current image is deleted and is not stored.
  • Step 570 storing the current image in response to ascertaining that the current image and the similar image are in different states.
  • the above executing body ascertains that the current image and the similar image are in the different states by comparing the intersection-over-union and the comparison result with the preset threshold value. It indicates that the image having the layout result of the current image is not stored in the local database, and thus, the current image is stored into the local database.
  • the current image is deleted or stored through the result of the state comparison, thus implementing different processing operations on the current image, and improving the diversity of the processing for the current image and the applicability of the layout result.
  • Fig. 6 illustrates a flow 600 of another embodiment of the method for detecting a layout.
  • the method for detecting a layout may further include the following steps.
  • Step 610 using an instance segmentation model to recognize a current image including at least one component in response to acquiring the current image, to obtain bounding box information and a category name of each component.
  • step 610 is the same as step 210 in the embodiment shown in Fig. 2, which will not be repeatedly described here.
  • Step 620 generating a marked image corresponding to the current image based on the bounding box information of the each component.
  • step 620 is the same as step 220 in the embodiment shown in Fig. 2, which will not be repeatedly described here.
  • Step 630 inputting the bounding box information and the category name of the each component and the marked image into a long short-term memory network detection model, and outputting a layout result of the component in the current image.
  • step 630 is the same as step 230 in the embodiment shown in Fig. 2, which will not be repeatedly described here.
  • Step 640 performing, for each component having a given layout, an adjustment operation on bounding box information of the each component to obtain adjusted bounding box information corresponding to the each component, in response to acquiring the layout result of the component in the current image.
  • the above executing body acquires the layout result of the component in the current image, and ascertains each layout (i.e., each grouping result) in the layout result, respectively.
  • Each grouping includes components having the given layout.
  • the adjustment operation is performed on the bounding box information of the each component, to align the bounding box information of the each component and set a width of the bounding box information of the each component to the same numerical value, thus obtaining the adjusted bounding box information corresponding to the each component.
  • the bounding box information of the each component in the given layout is adjusted to obtain the component aligned by group and having an average width, which improves the accuracy of the layout result.
  • a more accurate state comparison between images may be implemented based on the adjusted layout result.
  • the method for detecting a layout may further include the following steps.
  • Step 650 using the current image, the category name of the each component and the adjusted bounding box information as a new sample image set.
  • the above executing body uses the current image, the category name of the each component and the adjusted bounding box information as the new sample image set.
  • Step 660 training the instance segmentation model based on the new sample image set.
  • the above executing body may further train the instance segmentation model based on the new sample image set, to obtain an updated instance segmentation model.
  • the updated instance segmentation model may recognize a new image and output a category name sequence of each component in the new image and bounding box information of each component aligned by group and having an average width.
  • the instance segmentation model is trained through the new sample image set, such that the updated instance segmentation model can output the bounding box information of the each component aligned by group and having the average width, thereby improving the accuracy of the outputted bounding box information.
  • the method for detecting a layout may further include: inputting the adjusted bounding box information corresponding to the each component and the layout result into a UI2Code to obtain a duplicate layout code, in response to acquiring the adjusted bounding box information corresponding to the each component.
  • the above executing body inputs the adjusted bounding box information corresponding to the each component and the layout result into the UI2Code.
  • the UI2Code performs a redetection, a classification and an extraction on the customized component in the inputted content, and then applies a method of connecting a line between components, to ascertain a structured layout of each component, thereby obtaining a region having a duplicate layout between the components. Then, the duplicate layout code is generated using the obtained region having the duplicate layout between the components.
  • the adjusted bounding box information and the layout result are inputted into the UI2Code, thus improving the accuracy of the duplicate layout code generated by the UI2Code, and improving the reusability of the code.
  • the present disclosure provides an embodiment of an apparatus for detecting a layout.
  • the embodiment of the apparatus corresponds to the embodiment of the method shown in Fig. 2.
  • the apparatus may be applied in various electronic devices.
  • the apparatus 700 in this embodiment comprises: a recognizing module 710, a generating module 720 and an outputting module 730.
  • the recognizing module 710 is configured to use an instance segmentation model to recognize a current image including at least one component in response to acquiring the current image, to obtain bounding box information and a category name of each component.
  • the generating module 720 is configured to generate a marked image corresponding to the current image based on the bounding box information of the each component.
  • the outputting module 730 is configured to input the bounding box information and the category name of the each component and the marked image into a long short-term memory network detection model and output a layout result of the component in the current image, the layout result including sorting and grouping results of the each component in the current image.
  • the outputting module comprises: an encoding unit, configured to input the bounding box information and the category name of the each component into an encoding model in the long short-term memory network detection model, and output text feature information corresponding to the each component; an image processing unit, configured to input the marked image into an image processing model in the long short-term memory network detection model, and output image feature information corresponding to the marked image; and a decoding unit, configured to input the text feature information and the image feature information into a decoding model in the long short-term memory network detection model, and output the layout result of the component in the current image.
  • the instance segmentation model is acquired by: acquiring a sample image set, wherein the sample image set includes at least one sample image, and each sample image includes at least one component; ascertaining and annotating a category name and bounding box information of each component in the each sample image; and using the each sample image as an input, and using the category name and the bounding box information of the each component in the each sample image as a desired output, to train and obtain the instance segmentation model.
  • the apparatus further comprises: a searching module, configured to perform a state similarity search on stored images based on the layout result of the component in the current image, to obtain a plurality of similar images; and a determining module, configured to compare the current image with each similar image to determine whether the current image and the each similar image are in a given state.
  • a searching module configured to perform a state similarity search on stored images based on the layout result of the component in the current image, to obtain a plurality of similar images
  • a determining module configured to compare the current image with each similar image to determine whether the current image and the each similar image are in a given state.
  • the determining module comprises: a calculating unit, configured to calculate an intersection-over-union between the layout result of the component in the current image and a layout result of a component in the each similar image; a comparing unit, configured to compare a pixel of the current image with a pixel of the each similar image to obtain a comparison result between the pixel of the current image and the pixel of the each similar image; and a determining unit, configured to compare the intersection-over-union and the comparison result with a preset threshold value to determine whether the current image and the each similar image are in the given state.
  • the apparatus further comprises: a deleting module, configured to delete the current image in response to ascertaining that the current image and the similar image are in the given state; and a storing module, configured to store the current image in response to ascertaining that the current image and the similar image are in different states.
  • the apparatus further comprises: an adjusting module, configured to perform, for each component having a given layout, an adjustment operation on bounding box information of the each component to obtain adjusted bounding box information corresponding to the each component, in response to acquiring the layout result of the component in the current image.
  • an adjusting module configured to perform, for each component having a given layout, an adjustment operation on bounding box information of the each component to obtain adjusted bounding box information corresponding to the each component, in response to acquiring the layout result of the component in the current image.
  • the apparatus further comprises: an updating module, configured to use the current image, the category name of the each component and the adjusted bounding box information as a new sample image set; and a training module, configured to train the instance segmentation model based on the new sample image set.
  • the apparatus further comprises: a code generating module, configured to input the adjusted bounding box information corresponding to the each component and the layout result into a UI2Code to obtain a duplicate layout code, in response to acquiring the adjusted bounding box information corresponding to the each component.
  • the current image including the at least one component is recognized using the instance segmentation model in response to acquiring the current image, to obtain the bounding box information and the category name of the each component. Then, the marked image corresponding to the current image is generated based on the bounding box information of the each component. Finally, the bounding box information and the category name of the each component and the marked image are inputted into the long short-term memory network detection model, and the layout result of the component in the current image is outputted.
  • the layout result includes the sorting and grouping results of the each component in the current image, thus implementing that the layout result of the each component in the image is extracted.
  • the grouping result of the each component can be obtained based on the interdependence relationship between components. Accordingly, the global information between the portions of the current image is taken into consideration, thereby improving the accuracy of the layout result.
  • the above apparatus further comprises some other well-known structures, for example, a processor and a storage device.
  • these well-known structures are not shown in Fig. 7.
  • Fig. 8 is a schematic structural diagram of an electronic device (e.g., the terminal device in Fig. 1) 800 adapted to implement embodiments of the present disclosure.
  • an electronic device e.g., the terminal device in Fig. 1
  • the electronic device 800 may include a processing apparatus (e.g., a central processing unit and a graphic processing unit) 801, which may execute various appropriate actions and processes in accordance with a program stored in a read-only memory (ROM) 802 or a program loaded into a random access memory (RAM) 803 from a storage apparatus 808.
  • the RAM 803 also stores various programs and data required by operations of the electronic device 800.
  • the processing apparatus 801, the ROM 802 and the RAM 803 are connected to each other through a bus 804.
  • An input/output (I/O) interface 805 is also connected to the bus 804.
  • an input apparatus 806 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer and a gyroscope
  • an output apparatus 807 including, for example, a liquid crystal display (LCD), a speaker and a vibrator
  • a storage apparatus 808 including, for example, a magnetic tape and a hard disk
  • the communication apparatus 809 may allow the electronic device 800 to perform a wireless or wired communication with another device to exchange data.
  • Fig. 8 shows the electronic device 800 having various apparatuses, it should be understood that it is not required to implement or possess all of the illustrated apparatuses. More or fewer apparatuses may be alternatively implemented or possessed.
  • Each block shown in Fig. 8 may represent one apparatus, or may represent a plurality of apparatuses as needed.
  • an embodiment of the present disclosure includes a computer program product, including a computer program hosted on a computer readable medium, the computer program including program codes for performing the method as illustrated in the flowchart.
  • the computer program may be downloaded and installed from a network via the communication apparatus 809, may be installed from the storage apparatus 808, or may be installed from the ROM 802.
  • the computer program when executed by the processing apparatus 801, implements the above mentioned functionalities defined in the method according to the embodiments of the present disclosure.
  • the computer readable medium in the present disclosure may be a computer readable signal medium, a computer readable storage medium, or any combination of the two.
  • the computer readable storage medium may be, but not limited to: an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the above.
  • a more specific example of the computer readable storage medium may include, but not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read only memory (ROM), an erasable programmable read only memory (EPROM or flash memory), a fibre, a portable compact disk read only memory (CD-ROM), an optical memory, a magnet memory or any suitable combination of the above.
  • the computer readable storage medium may be any physical medium containing or storing programs, which may be used by a command execution system, apparatus or device or incorporated thereto.
  • the computer readable signal medium may include a data signal that is propagated in a baseband or as a part of a carrier wave, which carries computer readable program codes. Such propagated data signal may be in various forms, including, but not limited to, an electromagnetic signal, an optical signal, or any suitable combination of the above.
  • the computer readable signal medium may also be any computer readable medium other than the computer readable storage medium.
  • the computer readable medium may send, propagate or transmit programs for use by, or used in combination with, the command execution system, apparatus or device.
  • the program codes contained on the computer readable medium may be transmitted with any suitable medium including, but not limited to, a wire, an optical cable, an RF, or any suitable combination of the above.
  • the above computer readable medium may be the computer readable medium included in the above electronic device, or a stand-alone computer readable medium not assembled into the electronic device.
  • the above computer readable medium carries one or more programs.
  • the one or more programs when executed by the electronic device, cause the electronic device to: acquire a current image including at least one component, and use an instance segmentation model to recognize the current image, to obtain bounding box information and a category name of each component; generate a marked image corresponding to the current image based on the bounding box information of the each component; and input the bounding box information and the category name of the each component and the marked image into a long short-term memory network detection model, and output a layout result of the component in the current image, the layout result including sorting and grouping results of the each component in the current image.
  • a computer program code for executing the operations according to the embodiments of the present disclosure may be written in one or more programming languages or a combination thereof.
  • the programming language includes an object-oriented programming language such as Java, Smalltalk and C++, and further includes a general procedural programming language such as "C" language or a similar programming language.
  • the program codes may be executed entirely on a user computer, executed partially on the user computer, executed as a standalone package, executed partially on the user computer and partially on a remote computer, or executed entirely on the remote computer or a server.
  • the remote computer When the remote computer is involved, the remote computer may be connected to the user computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or be connected to an external computer (e.g., connected through Internet provided by an Internet service provider).
  • LAN local area network
  • WAN wide area network
  • an external computer e.g., connected through Internet provided by an Internet service provider.
  • each of the blocks in the flowcharts or block diagrams may represent a module, a program segment, or a code portion, the module, the program segment, or the code portion comprising one or more executable instructions for implementing specified logic functions.
  • the functions denoted by the blocks may occur in a sequence different from the sequences shown in the figures. For example, any two blocks presented in succession may be executed substantially in parallel, or they may sometimes be executed in a reverse sequence, depending on the function involved.
  • each block in the block diagrams and/or flowcharts as well as a combination of blocks in the block diagrams and/or flowcharts may be implemented using a dedicated hardware-based system executing specified functions or operations, or by a combination of dedicated hardware and computer instructions.
  • the modules involved in the embodiments of the present disclosure may be implemented by means of software or hardware.
  • the described modules may also be provided in a processor.
  • the processor may be described as: a processor comprising a recognizing module, a generating module and an outputting module.
  • the names of these modules do not in some cases constitute a limitation to such modules themselves.
  • the recognizing module may alternatively be described as "a module for using an instance segmentation model to recognize a current image including at least one component in response to acquiring the current image, to obtain bounding box information and a category name of each component.”

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • General Physics & Mathematics (AREA)
  • Evolutionary Computation (AREA)
  • Artificial Intelligence (AREA)
  • Software Systems (AREA)
  • Multimedia (AREA)
  • Medical Informatics (AREA)
  • Databases & Information Systems (AREA)
  • General Health & Medical Sciences (AREA)
  • Data Mining & Analysis (AREA)
  • Health & Medical Sciences (AREA)
  • Computing Systems (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • General Engineering & Computer Science (AREA)
  • Evolutionary Biology (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Quality & Reliability (AREA)
  • Image Analysis (AREA)

Abstract

Des modes de réalisation de la présente divulgation concernent un procédé et un appareil pour détecter une disposition. Une image en cours comprenant au moins un élément est reconnue à l'aide d'un modèle de segmentation d'instance en réponse à l'acquisition de l'image en cours, afin d'obtenir des informations de boîte englobante et un nom de catégorie de chaque élément. Ensuite, une image marquée correspondant à l'image en cours est générée sur la base des informations de boîte englobante de chaque élément. Finalement, les informations de boîte englobante et le nom de catégorie de chaque élément et de l'image marquée sont entrés dans un long modèle de détection de réseau de mémoire à court terme, et un résultat de disposition de l'élément dans l'image en cours est sorti. Le résultat de la disposition comprend le tri et le regroupement des résultats de chaque élément dans l'image en cours, ce qui permet de mettre en œuvre l'extraction du résultat de la disposition de chaque élément dans l'image. Le résultat de regroupement de chaque élément peut être obtenu sur la base de la relation d'interdépendance entre les éléments. Par conséquent, les informations globales entre les parties de l'image en cours sont prises en considération, ce qui permet d'améliorer la précision du résultat de la disposition.
PCT/KR2021/016801 2020-12-28 2021-11-16 Procédé et appareil de détection de disposition WO2022145723A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202011576492.0 2020-12-28
CN202011576492.0A CN112651942B (zh) 2020-12-28 2020-12-28 布局检测方法和装置

Publications (1)

Publication Number Publication Date
WO2022145723A1 true WO2022145723A1 (fr) 2022-07-07

Family

ID=75363358

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/KR2021/016801 WO2022145723A1 (fr) 2020-12-28 2021-11-16 Procédé et appareil de détection de disposition

Country Status (2)

Country Link
CN (1) CN112651942B (fr)
WO (1) WO2022145723A1 (fr)

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20170315969A1 (en) * 2014-04-29 2017-11-02 Wix.Com Ltd. System and method for the creation and use of visually-diverse high-quality dynamic layouts
US20180349110A1 (en) * 2017-05-31 2018-12-06 Wipro Limited Method and layout identification system for facilitating identification of a layout of a user interface
US20190188465A1 (en) * 2017-12-18 2019-06-20 Capital One Services, Llc Recognizing text in image data
US20200089997A1 (en) * 2018-09-18 2020-03-19 Focal Systems, Inc. Product onboarding machine
US20200380060A1 (en) * 2019-05-28 2020-12-03 Wix.Com Ltd. System and method for integrating user feedback into website building system services

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106910195B (zh) * 2017-01-22 2020-06-16 北京奇艺世纪科技有限公司 一种网页页面布局监测方法及装置
CN110084257A (zh) * 2018-01-26 2019-08-02 北京京东尚科信息技术有限公司 用于检测目标的方法和装置
CN109801276B (zh) * 2019-01-14 2023-05-19 沈阳联氪云影科技有限公司 一种计算心胸比的方法及装置

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20170315969A1 (en) * 2014-04-29 2017-11-02 Wix.Com Ltd. System and method for the creation and use of visually-diverse high-quality dynamic layouts
US20180349110A1 (en) * 2017-05-31 2018-12-06 Wipro Limited Method and layout identification system for facilitating identification of a layout of a user interface
US20190188465A1 (en) * 2017-12-18 2019-06-20 Capital One Services, Llc Recognizing text in image data
US20200089997A1 (en) * 2018-09-18 2020-03-19 Focal Systems, Inc. Product onboarding machine
US20200380060A1 (en) * 2019-05-28 2020-12-03 Wix.Com Ltd. System and method for integrating user feedback into website building system services

Also Published As

Publication number Publication date
CN112651942A (zh) 2021-04-13
CN112651942B (zh) 2023-04-07

Similar Documents

Publication Publication Date Title
WO2020164281A1 (fr) Procédé d'analyse de formulaire basé sur l'emplacement et la reconnaissance de caractères, ainsi que support et dispositif informatique
WO2020180013A1 (fr) Appareil d'automatisation de tâche de téléphone intelligent assistée par langage et vision et procédé associé
WO2020107765A1 (fr) Procédé, appareil et dispositif de traitement d'analyse de déclaration, et support de stockage lisible par ordinateur
WO2014069755A1 (fr) Système et procédé permettant de fournir un service de recommandation de contenu
WO2020253112A1 (fr) Procédé d'acquisition de stratégie de test, dispositif, terminal et support de stockage lisible
WO2019112342A1 (fr) Appareil de reconnaissance vocale et son procédé de fonctionnement
WO2015065006A1 (fr) Appareil multimédia, système d'éducation en ligne et procédé associé pour fournir un contenu d'éducation
WO2020253115A1 (fr) Procédé, appareil et dispositif de recommandation de produit basés sur une reconnaissance vocale et support de stockage
WO2024091080A1 (fr) Procédé de génération de vidéo automatique, et serveur de génération de vidéo automatique
CN114490395A (zh) 测试处理方法、装置、设备及介质
WO2021045552A1 (fr) Dispositif électronique de synthèse d'image et son procédé de fonctionnement
WO2022145946A1 (fr) Système et procédé d'apprentissage de langue sur la base d'images de formation recommandées par intelligence artificielle et de phrases illustratives
WO2022145723A1 (fr) Procédé et appareil de détection de disposition
WO2021025495A1 (fr) Dispositif électronique et procédé de traitement d'entrée manuscrite associé
WO2020180008A1 (fr) Procédé de traitement de plans comprenant de multiples points d'extrémité et dispositif électronique appliquant ledit procédé
WO2016117854A1 (fr) Appareil d'édition de texte et procédé d'édition de texte sur la base d'un signal de parole
WO2019164196A1 (fr) Dispositif électronique et procédé de reconnaissance de caractères
WO2016036049A1 (fr) Programme informatique, procédé, système et appareil de fourniture de service de recherche
WO2019143161A1 (fr) Dispositif électronique et son procédé de traitement de mot-clé de recherche
WO2022244997A1 (fr) Procédé et appareil pour le traitement de données
CN113849415A (zh) 控件测试方法、装置、存储介质及电子设备
WO2016108677A1 (fr) Appareil et procédé de sortie de contenu vidéo
WO2021002650A1 (fr) Procédé et appareil de traitement des images de correspondance croisée basés sur l'apprentissage profond
WO2023085610A1 (fr) Procédé et dispositif électronique pour effectuer un apprentissage de modèle multitâche
WO2023145983A1 (fr) Système fournissant un service d'analyse statistique personnalisé et procédé pour faire fonctionner le système

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21915521

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 21915521

Country of ref document: EP

Kind code of ref document: A1