Web Community Dataset

Hyun Chul Lee
Allan Borodin
Leslie Goldsmith

The dataset presented on this page is intended for the use of researchers. Permission is given researchers to download and use this data with the following provisions: the dataset is for the free and fair use of all and not for resale; the dataset must be cited giving the names of the compiler and editor of the dataset.

As indicated in our paper, we extracted 5 different communities, namely Camry, iPod, Xbox, PlayStation, and Mustang from a set consisting of about 2.84 million blog and forum entries. The provided dataset contains parsed raw data of about 330,000 blog/forum entries, which are constructed by taking the union of entries that compose the Camry, iPod, Xbox, PlayStation and Mustang communities. We present only a small subset of the original data, due to the size of the full set used in our experiments.

The data is in the tar.gz format and it consists of:

• Raw data (about 330,000 blog/forum entries)
  • Data is split over 34 directories (named files0-files33). Each directory contains about 10,000 blog/forum entries, with names in the format fileXXX, and an index file, named index.
    • fileXXX contains: (1) title data (tagged as title), (2) keywords found on the page and their frequencies (tagged as keywords), (3) links (tagged as links), and (4) link text (tagged as link text)
    • index contains a list of file names and their associates URLs.
• Seeds
  • We provide good and bad seeds used for each community extraction task. Each directory (e.g PlayStation) contains a file named good_seeds (containing a list of good seeds used) and another file name bad_seeds (containing a list of bad seeds used).

Downloads:

• web_community_data.tar.gz – Raw data files and seeds (~2.4GB)
• web_community_data_url.tar.gz – Reduced dataset that contains only the URLs and seeds (~6.5MB)