Skip to content

Introduce new fork / merge possibilities in MemorySource and MemoryCache#873

Merged
dgeb merged 7 commits into
mainfrom
forks
Jul 26, 2021
Merged

Introduce new fork / merge possibilities in MemorySource and MemoryCache#873
dgeb merged 7 commits into
mainfrom
forks

Conversation

@dgeb

@dgeb dgeb commented Jul 26, 2021

Copy link
Copy Markdown
Member

The primary use case of MemorySource forking is to create an isolated source and cache, which can be modified and then merged back into its base, with all the associated operations coalesced.

Because the forked MemorySource is "just" another Source, updates to it are still async, which allows for those updates to be logged, interacted with through other sources, etc. through async processes. Until now, there has been no way to synchronously make changes to a fork and then merge those back to the base source, because only updates at the source level have been tracked.

Optional update tracking in MemoryCache

This PR introduces a new optional form of update tracking in the cache associated with a source. By default now, this capability will be enabled in forked caches. Any updates applied to the cache will be tracked and used to form the merge transform when merge is invoked. This means that synchronous changes could be made to a forked cache directly rather than going through the associated source. Changes could also still be applied via updates to the forked source, since those will internally also invoke update on the cache and be tracked.

// fork a base source
let fork = source.fork();

// add jupiter synchronously to the forked source's cache
fork.cache.update((t) => t.addRecord(jupiter));

// merge changes from the fork back to its base
await source.merge(fork);

// jupiter should now be in the base source
source.cache.getRecordSync({ type: 'planet', id: 'jupiter' }); // returns jupiter

If you want to continue to track changes only at the source-level and have merge work only with those changes, pass the following config when you fork a source:

let fork = source.fork({ cacheSettings: { trackUpdateOperations: false } });

This will prevent update tracking at the cache level and will signal to merge that only transforms applied at the source-level should be merged.

Other new MemoryCache capabilities

Along with change tracking at the cache-level, this PR also introduces the following methods to MemoryCache:

  • fork - creates a new cache based on this one.
  • merge - merges changes from a forked cache back into this cache.
  • rebase - resets this cache's state to that of its base and then replays any update operations.

MemoryCache forking / merging / rebasing is a lighter-weight way of "branching" changes, that can ultimately be merged back into a source. Cache-level forking can be paired with source-level forking for a lot of flexibility and power.

dgeb added 7 commits July 24, 2021 16:39
Simplify rebasing by calling `this.cache.reset(base.cache)` Instead of resetting the cache state by updating the cache with inverse operations.
…`, `transformsSince` in favor of `getTransformsSince`

Provide naming consistency with other `getX` methods.
Allow caches to track ops that are applied via `update`. This will be done by default when a `base` cache is set. Otherwise, the `trackUpdateOperations: true` configuration setting is required to enable this feature.
…tracked

If a cache is tracking update operations, which is now the default for forked caches, then when the associated source is merged, the cache's operations will be applied in a single transform.

The benefit here is that it allows caches to be manipulated synchronously, and to have all those updates applied during a merge.

If you want to continue to merge only transforms applied to a source, and not its cache, then create the fork with `source.fork({ cacheSettings: { trackUpdateOperations: false } });`.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant