New System Maps Cross-Platform Code Forks That GitHub Can't Track

A new dataset released by researchers provides a corrected view of how code repositories relate to each other across forking platforms, addressing a longstanding blind spot in open-source software tracking.

The Problem with Counting Forked Repositories#

When developers fork code repositories, they preserve the entire git history. This means the same commits appear in dozens or hundreds of repositories simultaneously. Traditional metrics that count repositories directly become heavily inflated by these redundancies—sometimes by orders of magnitude—because they treat each fork as an independent project rather than a variant of the same work.

How the Solution Works#

The team created p2PFull, a deforking map built from the World of Code (WoC) version V2604 dataset. The approach works by:

• Identifying all 51.79 million shared-commit groups (clusters of repositories with overlapping git history)
• Using a "hub-node star encoding" and parallel Louvain clustering to group repositories into their true projects
• Applying optional size caps (cap250/cap500) to prevent boilerplate code and vendored libraries from artificially merging unrelated projects into giant clusters

The uncapped naive approach creates problems: the largest single cluster contained 443,010 repositories, bridged together only through shared libraries. Applying intelligent size limits removes these false connections while preserving genuine fork relationships.

Validation and Cross-Platform Discovery#

The researchers validated their map against GitHub's official fork graph and found 99.01% edge agreement. The remaining disagreements revealed something significant: 5.53% of fork families span multiple code forges, and 1.54% have their fork root outside GitHub entirely—relationships that GitHub's platform-specific graph cannot represent.

The team also released a refreshed fork-exclusion list with 134.1 million child repositories (3.4 times larger than previous catalogs) and a dataset of 455,550 detached forks with genuine independent origins.

Why It Matters#

Accurate fork genealogies are essential for understanding software development patterns, measuring code reuse, and tracking how innovations spread through open-source ecosystems. By recovering cross-forge relationships that existing platforms miss, this map enables more precise analysis of how code actually evolves across the internet.