Distributed Search Engine based on Nostr Protocol
Context
The Nostr protocol has rapidly emerged as a foundation for decentralized, censorship-resistant communication. Its lightweight, open design enables anyone to run a relay or client, fostering a global, permissionless network. Extending Nostr’s principles to search infrastructure unlocks the possibility of a truly distributed search engine—where billions of mobile devices collaboratively crawl, index, and serve web content without reliance on centralized servers. This approach promises not only greater privacy and resilience, but also customizable ranking and open participation, fundamentally reimagining how information is discovered and accessed online.
Motivation
Large tech companies, particularly Google, hold significant influence over our daily lives. Their recommendation systems have become highly sophisticated at understanding our preferences and behaviors. It’s important for us to regain more control over our personal data and digital experiences. A decentralized search engine would be a critical step in this direction, as it would shift power away from these corporations and back to individuals, fostering a more open and resilient information ecosystem.
Goal
Explore the design of a decentralized search engine where query routing, parsing and indexing are offloaded to a large population Nostr relays. The goal is to get a simple python prototype running that addresses the distributed indexing of the web and handling/routing of search queries. The clients should consist of users that want to get a list of search results and in return provides resources. Here different potential monetization/incentivization mechanisms should be explored. Examples could be payments via services like Bitcoin Lightning, Ad-based search, or lending ones own compute power to index the web or make it more secure.
This project description is just an initial outline and adaptations on what part the student wants to focus will be discussed.
Deliverables
- Architectural Design (max 8 pages)
- Design Principles
- Metric analysis: Scalability, Availability, Speed/Low cost
- Open issues/limitation analysis
- Python Code
- Presentation
Requirements
- Distributed systems fundamentals: knowledge of blockchains and Nostr
- Basic concepts of search engine (inverted indices, fuzzy search, vector search)
- Python
Pointers
- Y. Wei and G. Tyson, “ Exploring the Nostr Ecosystem: A Study of Decentralization and Resilience”, Feb. 08, 2024 (preprint).
- Nostr Protocol ( https://nostr.how/en/the-protocol)
- C. D. Manning, P. Raghavan, and H. Schütze (2008). “ Introduction to Information Retrieval”, Cambridge University Press
- Jyoti, “ Designing a Distributed Search System”, Medium, Accessed: Aug. 27, 2025.
- M. Herrmann, K. Ning, C. Diaz, and B. Preneel. “ Description of the YaCy Distributed Web Search Engine",
- M. Gregoriadis, R. Chotkan, P. Neague, and J. Pouwelse, Y. (2024). “ SwarmSearch: Decentralized Search Engine with Self-Funding Economy”, May 12, 2025
- Presearch. (n.d.). Node Overview. Retrieved from https://docs.presearch.io/nodes/overview