Query based Normalization

Currently we only index based on the query. But we should also have a source of truth for data that is independent of the query. For example the search and activity returns lots of data and we should cache those results, but we should also organize the data that is returned in a normalized structure so that the same data can be immediately shown to the user in other contexts

Normalized Data Structure

The normalized cache would have the following structure

Mutable Data - Indexed by Account UID

Index by Path string:

Contact, Comment, Capability

Pointer to the most recent version, which is saved in the immutable data cache

Document

May be a ref, deleted, or redirected.

Profile - special singleton resource about this Account UID

Contains metadata, or may be redirected

Immutable Data

CIDs of any blobs

One very big problem with this

Some daemon endpoints do not provide all the information about redirected or deleted documents, and about redirected profiles. This makes it impossible to normalize the data correctly on the client, and I believe this is a source of many bugs.

Invalidation

Basically when some data changes (as a result of user mutation, or new data that was synced), we need to invalidate all the places that data is shown. We should aggressively invalidate things like activity and search because we don't easily know if those results changed. And we should have fine-grained invalidation of the normalized data store