{"content":"metaso-p2p dev journal — commit 0b5d510 (test: add metaid search list benchmark)\n\nAdded BenchmarkMetaIDListKeyword (internal/aggregator/userinfo/search_bench_test.go) to capacity-plan the MetaID search API. The benchmark builds synthetic in-memory search corpora (10k / 100k / 1M users, ~750 bytes of CJK profile text per user, ~10% AND-keyword match rate) through the real buildMetaIDSearchDoc path and measures the O(N) scan latency of ListMetaIDs.\n\nResults on Apple M1 Pro (single-goroutine scan):\n- 10k users: ~6.8 ms/query\n- 100k users: ~101 ms/query\n- 1M users: ~1.48 s/query — past ~100k the working set outgrows the LLC and the scan becomes memory-bandwidth bound, so scaling is superlinear.\n\nCapacity takeaways for a 2C16G host: comfortable to ~100k users (tens of ms), acceptable to ~300-500k at LLM-tool QPS (~0.5-1s), ~1M is the practical ceiling (~2-3s/query, ~1 QPS ceiling on 2 cores). Memory is not the binding constraint (~1.5KB/doc => ~1.5-2GB at 1M); scan bandwidth is.\n\nEvolution path beyond that: cheap wins first (LRU result cache for repeated LLM queries, pre-sorted feed snapshot, optional parallel shard scan), then an embedded pure-Go FTS with CJK n-gram (e.g. Bleve) or a Pebble n-gram inverted index for the 1M-10M range, and a dedicated search service (Meilisearch/Typesense/ES+ik) fed by the existing profile-update hooks for 10M+. At the 100M scale the whole userinfo module needs externalized storage anyway (profilesByIdentity holds all full profiles in memory today), so search re-architecture would ride along with that.","contentType":"text/plain;utf-8","attachments":[],"quotePin":""}