Schemas for web data: a reverse engineering approach

摘要

In this paper, we show how to generate schemas of a set of HTML or XML documents retrieved from the web in the context of our web warehousing system called Whoweda (WareHouse Of WEb DAta). Web schemas are used to bind a web table that contains a collection of interlinked web documents called web tuples. These schemas specify the metadata, content and structural properties (in the form of predicates) shared by the web documents and hyperlinks in the web table. They also summarize the hyperlink structure of these documents using the notion of connectivities. Web schemas are generated in three stages. In the first stage, a simple or complex web schema is generated from the user's query (coupling query). In the next stage, the complex web schema is decomposed into a set of simple web schemas. These two stages are performed without inspecting the data instances, i.e., web tuples. Finally, in the last stage the set of simple web schemas are pruned by inspecting the hyperlink structure of the web tuples. We also discuss the formal algorithm for generating a set of simple web schemas from a coupling query.