Merging Adjacency Lists for Efficient Web Graph Compression

DOI: 10.1007/978-3-642-23169-8_42 In book: Man-Machine Interactions 2 AISC 103, pp.385-392

ABSTRACT Analysing Web graphs meets a difficulty in the necessity of storing a major part of huge graphs in the external memory, which
prevents efficient random access to edge (hyperlink) lists. A number of algorithms involving compression techniques have thus
been presented, to represent Web graphs succinctly but also providing random access. Our algorithm belongs to this category.
It works on contiguous blocks of adjacency lists, and its key mechanism is merging the block into a single ordered list. This
method achieves compression ratios much better than most methods known from the literature at rather competitive access times.

Keywordsgraph compression–random access

  • [Show abstract] [Hide abstract]
    ABSTRACT: Compressed representations have become effective to store and access large Web and social graphs, in order to support various graph querying and mining tasks. The existing representations exploit various typical patterns in those networks and provide basic navigation support. In this paper, we obtain unprecedented results by finding “dense subgraph” patterns and combining them with techniques such as node orderings and compact data structures. On those representations, we support out-neighbor and out/in-neighbor queries, as well as mining queries based on the dense subgraphs. First, we propose a compression scheme for Web graphs that reduces edges by representing dense subgraphs with “virtual nodes”; over this scheme, we apply node orderings and other compression techniques. With this approach, we match the best current compression ratios that support out-neighbor queries (i.e., nodes pointed from a given node), using 1.0–1.8 bits per edge (bpe) on large Web graphs, and retrieving each neighbor of a node in 0.6–1.0 microseconds ( $\upmu $ s). When supporting both out- and in-neighbor queries, instead, our technique generally offers the best time when using little space. If the reduced graph, instead, is represented with a compact data structure that supports bidirectional navigation, we obtain the most compact Web graph representations (0.9–1.5 bpe) that support out/in-neighbor navigation; yet, the time per neighbor extracted raises to around 5–20 $\upmu $ s. We also propose a compact data structure that represents dense subgraphs without using virtual nodes. It allows us to recover out/in-neighbors and answer other more complex queries on the dense subgraphs identified. This structure is not competitive on Web graphs, but on social networks, it achieves 4–13 bpe and 8–12 $\upmu $ s per out/in-neighbor retrieved, which improves upon all existing representations.
    Knowledge and Information Systems · 2.23 Impact Factor

Full-text (2 Sources)

Available from
May 21, 2014