Setup a model layer with the instance store and caching layer synchronized

[pmgmendes]

Hi.
I’m need to setup a model layer where a Type’s connection has the following capabilities:

1. when a new instance is saved (or an existing instance changed) it’s added (if not present) to the instanceStore and the server’s response data updates this instance;
2. when a define map’s prop is declared as that Type, the new instance(s) are returned from the instanceStore based in the unique identifier match;
3. when a instance is fetched from the server using Type.get({id: “A”}), the instance is first returned from the instanceStore and if server’s response data returns an new/modified props the instance is updated;

I’ve done some reading on the can-connect’s docs/tests/code to understand what behaviors to use and how to configure them.

The superMap connection at jsbin relies on fall-through-cache and it matches part of the desired capabilities: a cached instance is returned immediately with the previously created data; request to server is still done; the changed data from the server response forces the instance to updateaccordingly.
The unfulfilled expectation is the cached instance to be different from the previously saved instance.
It seems that the cached data was used to create and returned a new instance.

The custom connection at glitch make use of the behaviors that somehow seem to interact with the instanceStore.
There’s no cache connection setup in order to evaluate if the instanceStore is being filled and accessed as expected.
Here, the instanceStore its just added with the response of the latest Type.get() and the new initialization doesn’t go to the instanceStore to look for a instance with the same iD.
Note: I didn’t put it in jsbin too because the can.all.js distribution doesn’t have the can-connect/can/map/map behavior exported.

To wrap up, my expectation is to setup a connection in such a way that there’s a instance(s) store synchronized with the configured caching layer (localStorage or memoryCache).

How to achieve this?

Thanks!

[justinbmeyer]

.#1 on its own might create a memory leak. I think you want things only in the instanceStore if they are bound. This should happen by default with baseMap.

.#2 checkout the can/ref behavior. https://canjs.com/doc/can-connect/can/ref/ref.html This does what you want. You can use https://canjs.com/doc/can-construct.ReturnValue.html to set this up too.

.#3 I’m not sure how this differs from the fall-through cache. I haven’t looked at the glitch yet. Might have time tomorrow.

[pmgmendes]

1. These instances are bound in multiple places. I’ve updated the test in glitch to use baseMap and then declared my own connection with the same behaviors as baseMap. No difference in the outcome in both situations. Maybe its something in the test setup (fixtures, promise chaining, ?) and not the connection definition itself.
2. Thanks for the hint.
3. The fall-through-cache strategy works fine regarding the flow for return immediately » fetch from server to see if there’s any modified content » yes? then update the instance. But I am expecting the cache to return the same instance and not a new one.

newTodo.save().then(savedTodo => {
  assert.equal(savedTodo._cid, newTodo._cid); // OK
  Todo.get({id: "A"}).then(t => {
    assert.equal(t._cid, newTodo._cid); // NOK
    setTimeou(() => {
      assert.equal(t._cid, newTodo._cid); // still NOK
      assert.deepEqual(t.get(), newTodo.get()); // OK
    }, 1000); // add a delay so that the fixture can resolve
  });
});

The use cases for this kind of setup could described as:

• component A fetches, creates and updates Todo instances on the server;
• component A and B have multiple Todo.List props fetched from the server;
• component C and D have a Todo prop fetched from the server;

Whenever a existing Todo instance is updated in component A, the Todo or Todo.List props in components A, B, C and D that have a reference to that Todo instance must be updated thus forcing updates on every views where the instances or lists are bound and render.

Whenever a new Todo instance is created in component A, the Todo.List props in components A, B, C and D identified by a set/subset based on the filtering params will be augmented with this new Todo thus forcing updates on every views where these lists are bound and render.

Don’t if it’s the most efficient way but just because there’s some really extensive paths in the component tree to go through so that component A is able to notify components B, C or D that they need to re-fetch their data, I’m trying to use the model layer to keep the those views up to date.

[justinbmeyer]

Regarding

assert.equal(t._cid, newTodo._cid); // NOK

of:

newTodo.save().then(savedTodo => {
  assert.equal(savedTodo._cid, newTodo._cid); // OK
  Todo.get({id: "A"}).then(t => {
    assert.equal(t._cid, newTodo._cid); // NOK
    setTimeou(() => {
      assert.equal(t._cid, newTodo._cid); // still NOK
      assert.deepEqual(t.get(), newTodo.get()); // OK
    }, 1000); // add a delay so that the fixture can resolve
  });
});

This should not be happening. Is newTodo getting “A” assigned as its ID? What happens with:

newTodo.save().then(savedTodo => {
  assert.equal(savedTodo._cid, newTodo._cid); // OK
  savedTodo.id //=> ????
  Todo.get({id: savedTodo.id).then(t => {
    assert.equal(t._cid, newTodo._cid); // SHOULD BE OK
  });
});

Is newTodo bound to? It needs to be for this to work.

[justinbmeyer]

Looks like there is a bug: http://jsbin.com/midufa/edit?html,js

[justinbmeyer]

Reported bug here: https://github.com/canjs/can-connect/issues/296

[justinbmeyer]

Fixed here: https://github.com/canjs/can-connect/pull/297

Please check out those changes and let me know if it fixes your problem.

[justinbmeyer]

@pmgmendes Did you get a chance to checkout my fixes?

[pmgmendes]

@justinbmeyer Sorry the late reply.
I’m not able to get my test in glitch to pass.
I’ve updated the glitch project to can-connect from master since you’ve already merged the fix.
Does it make any difference the fixtures not being initialized through a store?

[justinbmeyer]

you have to listen on newTodo, not on something that points to it. The following binding:

app.on("todo", (ev, t) => {
    // check constructor-hydrate behavior
    assert.equal(t._cid, newTodo._cid, "app's todo instance is the same newTodo instance");
    assert.equal(t.version, 1, "because the app's todo is retrieved from the instanceStore the version 1");
  });

won’t do it.

[pmgmendes]

@justinbmeyer Yeah, now I’m listen to newTodo and it works great.
I’ve manually setup Todo’s connection instead of using superMap helper to see if I was able to use hydrate behavior.
I works great too. I’m putting the full test code here for future reference.
Thanks for the help!

import QUnit from "steal-qunit";
import fixture from "can-fixture";

import DefineMap from "can-define/map/";
import DefineList from "can-define/list/";
import set from "can-set/src/set";

import baseMap from "can-connect/can/base-map/";
import superMap from "can-connect/can/super-map/";
import connect from "can-connect";
import map from "can-connect/can/map/map";
import constructor from "can-connect/constructor/";
import constructorStore from "can-connect/constructor/store/";
import callbackOnce from "can-connect/constructor/callbacks-once/";
import dataCallbacks from "can-connect/data/callbacks/";
import dataParse from "can-connect/data/parse/";
import dataUrl from "can-connect/data/url/";
import merge from "can-connect/can/merge/";
import hydrate from "can-connect/can/constructor-hydrate/constructor-hydrate";
import callbacksCache from "can-connect/data/callbacks-cache/";
import fallThroughCache from "can-connect/fall-through-cache/";
import localStorageCache from "can-connect/data/localstorage-cache/";
import combineRequests from "can-connect/data/combine-requests/combine-requests";
import callbacksOnce from "can-connect/constructor/callbacks-once/callbacks-once";
import ref from "can-connect/can/ref/";
import realTime from "can-connect/real-time/";
import cacheRequests from "can-connect/cache-requests/";


QUnit.module("Todo module", {
  beforeEach: () => {
    localStorage.clear();
  }
});

QUnit.test("Does cache returns the same instance", function(assert) {

  fixture({
    "POST /todos": {
      "id": "A",
      "name": "overwritten by the server",
      "version": 1
    },
    "GET /todos/A": {
      "id": "A",
      "name": "new todo",
      "version": 2
    }
  });
  
  
  const Todo = DefineMap.extend({
    id: "string",
    name: "string",
    version: "number"
  });

  Todo.List = DefineList.extend({
    "#": Todo
  });

  Todo.algebra = new set.Algebra(
    set.props.id("id")
  );

  Todo.connection = superMap({
    Map: Todo,
    List: Todo.List,
    url: "/todos",
    name: "todo",
    algebra: Todo.algebra
  });
  
  Todo.connection = connect([
    constructor,
    map,
    //ref,
    constructorStore,
    dataCallbacks,
    //combineRequests,
    dataParse,
    dataUrl,
    realTime,
    callbackOnce,
    callbacksCache,
    fallThroughCache,
    hydrate
    //merge,
    //cacheRequests
  ], {
    Map: Todo,
    List: Todo.List,
    url: "/todos",
    name: "todo",
    algebra: Todo.algebra,
    cacheConnection: connect([localStorageCache],{
      name: "todoCache",
      idProp: "id",
      algebra: Todo.algebra
    })
  });
    
  const Page = DefineMap.extend({
    todo: {Type: Todo}
  });
  
  let newTodo = new Todo({name: "user input"}), 
      page = new Page({});
  
  newTodo.on("id", function(){});
  
  let done = assert.async();
  
  newTodo.save().then(savedTodo => {
    
    assert.equal(savedTodo._cid, newTodo._cid, "createData returns the newTodo instance");
    assert.equal(savedTodo.name, "overwritten by the server", "the POST response overwrites the name");
    assert.equal(savedTodo.version, 1, "the server setted the version to 1");
    
    Todo.get({id: savedTodo.id}).then(t => {
      
      assert.equal(t._cid, newTodo._cid, "todo instance retrieved from instanceStore is the newTodo instance");
      assert.equal(t.name, "overwritten by the server", "same name");
      assert.equal(t.version, 1, "the server setted the version to 1");
      
      // hydrate behavior: a new instance of Todo with the same 'id' is returned from instanceStore
      page.todo = {id: "A"};
      assert.equal(page.todo._cid, newTodo._cid, "app.todo instance is the newTodo instance");
      
      setTimeout(() => {
        // fall-through-cache do it's thing
        assert.equal(t.version, 2, "todo from server, the version as changed");  
        done();
      }, 3000);
    });
    
  });
});