Hybrid System Predicts Both Common and Rare Behaviors on Bluesky Social Network

Forecasting what users will do on social media remains a significant challenge for platforms seeking to improve recommendation systems and design. Most prior work concentrates on high-frequency actions like retweets and likes, leaving the prediction of less common but meaningful behaviors largely unexplored.

Researchers have developed a layered prediction system designed to handle both frequent and infrequent user actions on the decentralized social network Bluesky. Their approach proved successful enough to win first place at the SocialSim competition hosted during the Conference on Language Modeling (COLM 2025).

The Data and Task#

The team worked with a substantial corpus drawn from Bluesky: 6.4 million conversation threads featuring 12 distinct types of user actions distributed across 25 different persona clusters. The task required the system to distinguish patterns not just among dominant behaviors but also among actions that occur rarely yet carry meaningful weight.

The Four-Part Architecture#

The solution integrates four complementary techniques:

• Lookup database: A pattern-matching system that retrieves predictions based on historical precedents in user response sequences
• LightGBM models: Machine learning classifiers trained separately for each persona, using engineered features capturing temporal patterns and semantic meaning; these handled frequent actions
• Hybrid neural module: A specialized deep-learning component combining text and time-series representations to classify rare behaviors
• Reply generation: An additional component capable of generating text responses

Performance Metrics#

For common actions, the persona-specific machine learning models achieved an average macro F1-score of 0.64—a standard measure of prediction accuracy balancing precision and recall. The rare-action classifier reached a 0.56 macro F1-score when evaluated across 10 less frequent action types.

Why It Matters#

These results highlight an important principle: effective prediction of user behavior demands different techniques for different action frequencies. The research suggests that treating all actions uniformly misses critical opportunities. Rare behaviors, while infrequent, often signal important user interests or platform dynamics that merit specialized modeling approaches rather than one-size-fits-all solutions.