If you want to read the PDF, try requesting it from the authors.

Abstract

Nowadays most of the patent search systems still rely upon text to provide retrieval functionalities. Recently, the intellectual property and information retrieval communities have shown great interest in patent image retrieval, which could augment the current practices of patent search. In this chapter, we present a patent image extraction and retrieval framework, which deals with patent image extraction and multimodal (textual and visual) metadata generation from patent images with a view to provide content-based search and concept-based retrieval functionalities. Patent image extraction builds upon page orientation detection and segmentation, while metadata extraction from images is based on the generation of low level visual and textual features. The content-based retrieval functionality is based on visual low level features, which have been devised to deal with complex black and white drawings. Extraction of concepts builds upon on a supervised machine learning framework realised with Support Vector Machines and a combination of visual and textual features. We evaluate the different retrieval parts of the framework by using a dataset from the footwear and the lithography domain.

No full-text available

Request Full-text Paper PDF

To read the full-text of this research,
you can request a copy directly from the authors.

Chapter
Improving the performance of the patented image retrieval system is of great significance in the intellectual property protection. The design patent image has a large amount of data, and how to quickly complete the retrieval is part of the main research issues for the design patent retrieval system. Classification is an effective way to improve the retrieval speed, so some methods of image classification have been proposed before. However, image classification cannot achieve high-level semantic classification. Thus the speed of improvement is very limited. In order to realize the classification effect of high-level semantics, in this paper, we propose a method that uses the image caption model-based to realize the automatic description generation of the design patent image. Experiments show that our method has better classification accuracy and better semantic classification performance than previous image classification methods.
Conference Paper
Full-text available
The aim of this document is to describe the methods we used in the Patent Image Classification and Image-based Patent Retrieval tasks of the Clef-IP 2011 track. The patent image classification task consisted in categorizing patent images into pre-defined categories such as abstract drawing, graph, flowchart, table, etc. Our main aim in participating in this sub-task was to test how our image categorizer performs on this type of categorization problem. Therefore, we used SIFT-like local orientation histograms as low level features and on the top of that we built a visual vocabularies specific to patent images using Gaussian mixture model (GMM). This allowed us to represent images with Fisher Vectors and to use linear classifiers to train one-versus-all classifiers. As the results show, we obtain very good classification performance. Concerning the Image-based Patent Retrieval task, we kept the same image repre-sentation as for the Image Classification task and used dot product as similarity measure. Nevertheless, in the case of patents the aim was to rank patents based on patent similarities, which in the case of pure image-based retrieval implies to be able to compare a set of images versus another set of images. Therefore, we investigated different strategies such as averaging Fisher Vector representation of an image set or considering the maximum similarity between pairs of images. Finally, we also built runs where the predicted image classes were considered in the retrieval process.
Conference Paper
Full-text available
This paper presents the evaluation of a number of algorithm alternatives for content based retrieval from a database of technical drawings representing patents. The objective is to help patent evaluators in their quest for a possible patent bearing too much similarity with the one under investigation. To achieve this, we have devised a system where images (drawings) are represented using attributed graphs based on the extracted line-patterns or histograms of attributes computed from the graphs. Retrieval is either performed using histogram comparison (see Huet, B. and Hancock, E.R., PAMI, vol.21, no.12, p.1363-70, 1999) or thanks to a graph similarity measure (see Huet and Hancock, CVPR, p.138-43, 1998). Promising results are presented along with possible work extension
Conference Paper
Full-text available
In this paper, we outline some of the main challenges facing trademark searchers today, and discuss the extent to which current automated systems are meeting those challenges.
Conference Paper
Full-text available
A patent always contains some images along with the text. Many text based systems have been developed to search the patent database. In this paper, we describe PATSEEK that is an image based search system for US patent database. The objective is to let the user check the similarity of his query image with the images that exist in US patents. The user can specify a set of key words that must exist in the text of the patents whose images will be searched for similarity. PATSEEK automatically grabs images from the US patent database on the request of the user and represents them through an edge orientation autocorrelogram. L1 and L2 distance measures are used to compute the distance between the images. A recall rate of 100% for 61% of query images and an average 32% recall rate for rest of the images has been observed.
Conference Paper
Full-text available
The combination of different search techniques can improve the results given by each one. In the ongoing R&D projec t PATExpert1, four different search techniques are combined to perform a patent search. These techniques are: metadata search, keyword-based sear ch, semantic search and image search. In this paper we propose a general ar chitecture based on web services where each tool works in its own domain an d provides a set of basic functionalities to perform the retrieval. To be abl e to combine the results from the four search engines, these must be fuzzy (using a membership function or similarity grade). We focus on how the fuzzy result s can be obtained from each technique, and how they can then be combined. This combination must take into account the query, the similarity of the paten t to each part of the query, and the confidence on the technique
Article
Full-text available
The aim of the SCHEMA Network of Excellence is to bring together a critical mass of universities, research centers, industrial partners and end users, in order to design a reference system for content-based semantic scene analysis, interpretation and understanding. Relevant research areas include: content-based multimedia analysis and automatic annotation of semantic multimedia content, combined textual and multimedia information retrieval, semantic -web, MPEG-7 and MPEG-21 standards, user interfaces and human factors. In this paper, recent advances in content-based analysis, indexing and retrieval of digital media within the SCHEMA Network are presented. These advances will be integrated in the SCHEMA module-based, expandable reference system.
Article
Full-text available
A training algorithm that maximizes the margin between the training patterns and the decision boundary is presented. The technique is applicable to a wide variety of classifiaction functions, including Perceptrons, polynomials, and Radial Basis Functions. The effective number of parameters is adjusted automatically to match the complexity of the problem. The solution is expressed as a linear combination of supporting patterns. These are the subset of training patterns that are closest to the decision boundary. Bounds on the generalization performance based on the leave-one-out method and the VC-dimension are given. Experimental results on optical character recognition problems demonstrate the good generalization obtained when compared with other learning algorithms. 1
Conference Paper
Trademark image searching is potentially one of the most important application areas for automated content-based image retrieval (CBIR) techniques. There are many large and growing collections of trademark images in electronic form. The task of maintaining manual indexes to these image collections is becoming increasingly onerous. And there is a paramount need for accurate and reliable searching, since the images can be of major commercial significance.
Article
The ever-increasing number of registered trademarks has created greater demand for an automatic trademark retrieval system. In this paper, we present a method for such a system based on the image content, using a shape feature. Zernike moments of an image are used for a feature set. To retrieve similarly shaped trademarks quickly, we introduced the concept of a `visually salient feature' that dominantly affects the global shape of the trademarks. Experiments have been conducted on a database of 3000 trademark images. The retrieval speed was very fast and similar-shaped trademark retrieval results were very promising.
Article
Taking as a starting point the actual and the perceived value of models of claimed inventions in the 19th century, the author explores the proposition that non-text disclosures of this type and, more significantly, their modern equivalents as 3D electronic images, as well as many other forms of electronic non-text material, will have an increasing place in fully communicating the inventions filed in 21st century patent applications. He argues that moving to a patent application process in which supplementary machine readable material can be supplied, and can form part of the total disclosure of an invention, without the need for it to be convertible to paper form, would increase intelligibility of invention disclosures of both existing problem areas such as genetic sequence patents, as well as forestalling increasing problems likely to arise in other areas as the 21st century unfolds. He notes that in this way the limitations inherent in having to describe 21st century inventions by using the 15th century technology of the printed page could be overcome. Analogies with the situation for certain types of trademark applications for registration, e.g. sound marks, are also noted.
Article
The automatic removal of suffixes from words in English is of particular interest in the field of information retrieval. An algorithm for suffix stripping is described, which has been implemented as a short, fast program in BCPL. Although simple, it performs slightly better than a much more elaborate system with which it has been compared. It effectively works by treating complex suffixes as compounds made up of simple suffixes, and removing the simple suffixes in a number of steps. In each step the removal of the suffix is made to depend upon the form of the remaining stem, which usually involves a measure of its syllable length.
Conference Paper
Recently, the intellectual property and information retrieval communities have shown increasing interest in image retrieval, which could augment the current practices of patent search. In this context, this article presents PatMedia search engine, which is capable of retrieving patent images in content-based manner. PatMedia is evaluated both by presenting results considering information retrieval metrics, as well as realistic patent search scenarios.
Article
LIBSVM is a library for support vector machines (SVM). Its goal is to help users to easily use SVM as a tool. In this document, we present all its imple-mentation details. For the use of LIBSVM, the README file included in the package and the LIBSVM FAQ provide the information.
Article
This paper describes an ongoing research project aimed at implementing a trademark retrieval system using an as-sociative memory neural network. The novel aspect presented in this paper is the proposed integrated framework for image retrieval using multiple representations of images based on gestalt principles. In this paper we summarise the methods we followed in extracting local perceptual features as well as features of the closed figures of images. In designing the search engine of the system we have adopted a novel similarity assessment criteria based on local features as well as features of the closed figures, which is being implemented using an associative memory neural network to achieve high performance in retrieval. Then we describe the strategy we followed in combining multiple similarity measures and present the results obtained from the first phase of evaluation of the system.
Article
In this article, we discuss the potential benefits, the requirements and the challenges involved in patent image retrieval and subsequently, we propose a framework that encompasses advanced image analysis and indexing techniques to address the need for content-based patent image search and retrieval. The proposed framework involves the application of document image pre-processing, image feature and textual metadata extraction in order to support effectively content-based image retrieval in the patent domain. To evaluate the capabilities of our proposal, we implemented a patent image search engine. Results based on a series of interaction modes, comparison with existing systems and a quantitative evaluation of our engine provide evidence that image processing and indexing technologies are currently sufficiently mature to be integrated in real-world patent retrieval applications.
Article
This paper proposes a method for binary image retrieval, where the black-and-white image is represented by a novel feature named the adaptive hierarchical density histogram, which exploits the distribution of the image points on a two-dimensional area. This adaptive hierarchical decomposition technique employs the estimation of point density histograms of image regions, which are determined by a pyramidal grid that is recursively updated through the calculation of image geometric centroids. The extracted descriptor combines global and local properties and can be used in variant types of binary image databases. The validity of the introduced method, which demonstrates high accuracy, low computational cost and scalability, is both theoretically and experimentally shown, while comparison with several other prevailing approaches demonstrates its performance.
Conference Paper
Just as in the real world, plants are important objects in virtual world for creating pleasant and realistic environments, especially those involving natural scenes. As such, much effort has been made in realistic modeling of plants. As the trend moves ...
Conference Paper
This report presents the work carried out for the image classification task in the course of the CLEF-IP 2011 competition. Based on the visual content, patent images are automatically classified into several drawing types, such as abstract drawings, tables, flow chart and graphs. For that purpose, a series of SVM classifiers, multi-modal fusion schemes and a variety of content-based low-level features for black and white images were used. The overall reported performance was promising. Our best runs achieved a true positive rate of over 66% and the reported average area under curve is over 0.9.
Article
With ever increasing number of registered trademarks, the task of trademark office is becoming increasingly difficult to ensure the uniqueness of all trademarks registered. Trademarks are complex patterns consisting of various image and text patterns, called device-mark and word-in-mark respectively. Due to the diversity and complexity of image patterns occurring in trademarks, due to multi-lingual word-in-mark, there is no very successful computerized operating trademark registration system. We have tackled key technical issues: multiple feature extraction methods to capture the shape, similarity of multi-lingual word-in-mark, matching device mark interpretation using fuzzy thesaurus, and fusion of multiple feature measures for conflict trademark retrieval. A prototype System for Trademark Archival and Registration (STAR) has been developed. The initial test run has been conducted using 3000 trademarks, and the results have shown satisfaction to trademark officers and specialists.
Article
Proper processing and efficient representation of the digitized images of printed documents require the separation of the various information types: text, graphics, and image elements. For most applications it is sufficient to separate text and nontext, because text contains the most information. This paper describes the implementation and performance of a robust algorithm for text extraction and segmentation that is completely independent of text orientation and can deal with text in various font styles and sizes. Text objects can be nested in nontext areas, and inverse printing can also be analyzed. It should be mentioned that the classification is based only on rough image features, and individual characters are not recognized. The three main processing steps of the system are the generation of connected components, neighborhood analysis, and generation of text lines and blocks. As output, connected components are classified as text or nontext. Text components are grouped as characters, words, lines, and blocks. Nontext objects are accumulated as a separate nontext block.
Article
In focusing on the field of searching mechanical and device patents, the author explores the available resources, their uses and their limitations. In contrast to searches in the field of chemical patents, this field does not usually provide the benefits of consistent terminology, deep level classification and indexing or direct searching of drawings (cf structural compound search in chemistry). The many challenging problems that result for the searcher and the state of the art in progress on direct searching of drawings are reviewed. Illustrative examples in the fields of medical inhalers and pump actuators are provided. It is clear that there is still a long way to go before practicable, searchable databases can be created or even one of the key initial steps in this process - the general filing and manipulation of 3D drawings in patent applications - can be realised. Methods of searching patent images and tools for faster evaluation of drawings in the full patent specification would be welcomed.
Conference Paper
Since there are mass of images in the outward-appearance patent-image database, a retrieval approach developed for this database is greatly demanded. The shape information of the patent image is the key feature for a patent image. Therefore, we present an outward-appearance patent images retrieval approach based on contour- description matrix. This matrix represents the 2D shape feature and shows the angular and radial distributing information. The proposed representation is invariant to scale, translation and rotation. Our experiments show that the contour-based technique performs significantly better compared to wavelet-based approach previously proposed in the literature. Key words: shape retrieval; patent image
Article
The Artisan system retrieves abstract trademark images by shape similarity. It analyzes each image to characterize key shape components, grouping image regions into families that potentially mirror human image perception, and then derives characteristic indexing features from these families and from the image as a whole. We have evaluated the retrieval effectiveness of our prototype system on more than 10,000 images from the UK Trade Marks Registry
Article
Retrieval efficiency and accuracy are two important issues in designing a content-based database retrieval system. We propose a method for trademark image database retrieval based on object shape information that would supplement traditional text-based retrieval systems. This system achieves both the desired efficiency and accuracy using a two-stage hierarchy: in the first stage, simple and easily computable shape features are used to quickly browse through the database to generate a moderate number of plausible retrievals when a query is presented; in the second stage, the candidates from the first stage are screened using a deformable template matching process to discard spurious matches. We have tested the algorithm using hand drawn queries on a trademark database containing 1; 100 images. Each retrieval takes a reasonable amount of computation time (¸ 4-5 seconds on a Sun Sparc 20 workstation). The top most image retrieved by the system agrees with that obtained by human subjects, ...
Mechanical Patent Searching: A Moving Target
  • De Marco
De Marco, D.: Mechanical Patent Searching: A Moving Target. In: Patent Information Users Group (PIUG), Baltimore, USA (2010)
Evaluation of Patent Image Retrieval
  • G Ypma
Ypma, G.: Evaluation of Patent Image Retrieval. In: Information Retrieval Facility Symposium 2010 (IRFS 2010), Vienna, Austria (2010)
Advanced Content-Based Semantic Scene Analysis and Information Retrieval: The Schema Project
  • E Izquierdo
  • J Casas
  • R Leonardi
  • P Migliorati
  • N Connor
  • I Kompatsiaris
  • M G Strintzis