noiano/commoncrawl
CommonCrawl Project Repository
Packages the ARCInputFormat used in Common Crawl in a small jar file that can be used in MapReduce jobs. Implements HdfsARCSource. See README for details
This repository is cataloged as part of our automated global GitHub synchronization. Full telemetry, velocity snapshots, and code summaries are scheduled for continuous enrichment.
CommonCrawl Project Repository
Simple utility to convert an existing compressed file in an Hadoop SequenceFile