[Solved] Struggling with CanJS packages between major releases

CanJS
1

I’m using the can package, as opposed to individual deps themselves, which poses an issue when the deps of the can package become outdated.

This very topic was discussed in the following contributors meeting:

One proposal was to use CI to have evergreen dependencies for the can package. That would be convenient and work most of the time, however I could see this being insufficient after a major version bump when a pinned version of a dependency is used indirectly by multiple other dependencies.

Take for example my experience with can-view-live:

$ npm ls can-view-live

my-project@0.0.1
└─┬ can@5.11.0
  ├─┬ can-stache@4.12.0
  │ └── can-view-live@4.2.1  deduped
  ├─┬ can-stache-bindings@4.4.0
  │ └── can-view-live@4.2.1  deduped
  └── can-view-live@4.2.1

Currently, the can package forces a fixed (older) version:

It seems the thinking is maintainers should know (and remember) to make a new release of the can package immediately after pushing a hotfix to a can package dependency. Unfortunately, it seems this hasn’t been the case for can-view-live to which I believe a hotfix was pushed ~20 hours ago. It is still not reflected in a new can package release. Luckily, those other packages that depend on can-view-live use a version range that includes the bump.

can-stache:

can-stache-bindings:

So, I thought to just pin the can package to the un-merged Greenkeeper PR branch:

$ npm i --save github:canjs/canjs#greenkeeper/can-view-live-4.2.2

+ can@5.11.0
added 1 package and updated 1 package in 14.767s

That worked to update the can-view-live dependency at the top level, but nested dependencies were still using the older version of can-view-live:

$ npm ls can-view-live

my-project@0.0.1
└─┬ can@5.11.0 (github:canjs/canjs#0ffc3526d14cb5e8c366203049e75c1dbc34367d)
  ├─┬ can-stache@4.12.0
  │ └── can-view-live@4.2.1
  ├─┬ can-stache-bindings@4.4.0
  │ └── can-view-live@4.2.1  deduped
  └── can-view-live@4.2.2

I found that if I wanted the nested deps to go from 4.2.1 to 4.2.2, it was necessary to blow away node_modules and re-run npm install:

$ rm -rf node_modules
$ npm install

This resulted in can-view-live@4.2.2 for all deps:

$ npm ls can-view-live

my-project@0.0.1
└─┬ can@5.11.0 (github:canjs/canjs#0ffc3526d14cb5e8c366203049e75c1dbc34367d)
  ├─┬ can-stache@4.12.0
  │ └── can-view-live@4.2.2  deduped
  ├─┬ can-stache-bindings@4.4.0
  │ └── can-view-live@4.2.2  deduped
  └── can-view-live@4.2.2

However, if I needed to update more than just the one can-view-live dependency, this Greenkeeper branch strategy would fail, as it only allows updating the version of one dependency at a time.

Looking around, I found this Stack Overflow post that mentions manipulating the package-lock.json:

However, this is not possible in my project because I have an .npmrc with package-lock=false and, as mentioned in this tool for automating the process, it’s a big hack that should be avoided anyways.

So, I guess it’s looking like if I had multiple dependencies that I needed to be updated in the can package, my best bet would be to fork the can repository and update those packages myself until a release of the can package was made that met my requirements.

I’m not sure if there is a better workflow, but it is something worth documenting, as dependency management within the plethora of interrelated modules in the Bitovi frameworks has probably been the biggest time-sink and non-starter for me when getting a project off the ground. I’m always having to manually patch things and cherry pick pull requests that have not been merged for months.

If you’re new, the sheer number of packages can be overwhelming in addition to the volume of unresolved issues and PRs that are often times unintelligible to a newbie outsider. It seems there are many new concepts being introduced with incremental rendering and dynamic component creation, and eventually things will settle down. However, I feel the huge number of un-merged Greenkeeper PRs for example make it hard for newcomers (or anyone, really) to wade through and find the active branches for instance.

In general, it would be nice if there was a push to have all packages updated so that deps are pinned to the latest versions across the board as a kind of fresh start, and to delete the mass of Greenkeeper branches in the repos to make the truth about dependencies more explicit and everything easier to navigate. Hopefully that is something that will come soon, in time.

2

In general, it would be nice if there was a push to have all packages updated so that deps are pinned to the latest versions across the board as a kind of fresh start

What would this solve? This would make it impossible to use custom versions which it sounds like you want to do.

Also with respect to:

and to delete the mass of Greenkeeper branches in the repos to make the truth about dependencies more explicit and everything easier to navigate

I’m not sure how the greenkeeper branches affects “dependency truth”. Each package has its own versions it can use. Each canjs release represents a “verified” integration of pinned dependencies.

If you’re new, the sheer number of packages can be overwhelming in addition to the volume of unresolved issues and PRs that are often times unintelligible to a newbie outsider.

For outsiders, they should only really use the can package with its hard-coded dependencies.

For “advanced” users, they should import packages directly as is suggested here: Setting Up CanJS | getting started | Guides | CanJS — Build CRUD apps in fewer lines of code.

This allows you to make your own version of can that avoids the boilerplate as noted in the Can I avoid that boilerplate? section.

Unfortunately, I don’t think there’s another “have your cake and eat it”. Either people wait for official “canjs” releases, or they build their own “can” and manage the individual dependencies themselves.

3

The other idea we had was to publish a “can5” package that would essentially use ^ on all deps. Folks could then use package-lock to control nested deps.

4

You’re linking to a guide that’s 3 months old… That would explain why I didn’t see it documented anywhere, because I haven’t gone through the Guides section in quite a while! :stuck_out_tongue_winking_eye:

I’m glad to see there is some very good documentation regarding can package deps after all. :+1:

Replace “pinned to the latest versions” with “updated to the latest versions, but still ranged”. This would solve the problem that un-merged Greenkeeper branches and outdated version ranges cause me more questions than they answer, especially when my project won’t build and I’m frustrated with the framework!

I think less Greenkeeper branch and PR noise would help reduce the complexity of piecing related repos together when there’s an issue. I meant “dependency truth” in regards to the frustration a user can feel when trying to sort out which version of a dependency is the currently acceptable (“true”) version, or at least the version they should try next when things aren’t working. Hope that makes sense.

Also, if you’re looking for a branch that might have the fix you need, it can be a struggle… Right now one has to go through four or five pages of outdated Greenkeeper branches to find that one semi-active branch that received a commit months ago.

This’s what I was doing when my application stopped working without any major changes from me. :angry:

Thanks, that’s exactly the guidance I needed! :heart: I’ll try importing packages directly.

  • Your suggestion in the Contributors video of having a CanJS builder tool sounded very intriguing.
  • It would also be cool if I could “eject” from using the can package without a bunch of hand-typing.
  • I might call the version of the can package with explicitly pinned deps the can5 (or can6 / can7 / can8) version, while just can is made to be the one that is ranged, evergreen, and essentially meant to be managed by a package-lock.json.

Concluding thoughts:

You might not see it, but from an outsider perspective there is a lot of complexity to the many interconnected CanJS repositories, especially when you’re also pulling in parts from StealJS and DoneJS.

Having packages be more explicit about their most current deps would instill confidence and remove guesswork when trying to piece together code someone else wrote.

I feel that managing the huge number of CanJS packages has been one of the worst pain points in using CanJS for me, especially because when starting a new project you want to use the latest and greatest!

I’m just thinking, the package repos may be made friendlier with less noise and more explicit deps. :thinking:

5

Why were you doing this?

Having packages be more explicit about their most current deps would instill confidence and remove guesswork when trying to piece together code someone else wrote.

Are they not already explicit about this? Why are they not? They point at the acceptable range of deps that pass their tests. This wide range is necessary if we are going to allow people the freedom to be pinned to a mix of old and new versions.

6

You might not see it, but from an outsider perspective there is a lot of complexity to the many interconnected CanJS repositories,

Yes, we do see this. This is why in CanJS 5, most users shouldn’t be aware of those repos. (Also, our plan is to change the docs to show the named exports instead of package names). Is the bug the only reason you had to become aware of the interconnected CanJS repos?

8

Thanks for asking, I’ll try and explain:

I recently had an issue with GitHub - donejs/autorender: Automatically render to the body or html element that eventually led me to the many, many Greenkeeper branches on the GitHub web UI for the repo. When I added done-autorender to my project it wasn’t working as expected, and I subsequently observed a helpful console warning. From that, I could more or less infer that there was an issue with the template bindings. Now, to try and resolve this issue on my own, I had to become aware of the interconnected CanJS repos. The first thing I thought to do was to check if there was a newer release not yet published to NPM. Using the releases tab on GitHub, I could see there had been no release made for almost a month. So, I moved on to pointing done-autorender to the GitHub master branch github:donejs/autorender#master. This didn’t resolve the issue, either, so my next troubleshooting step was to see if there was any pending branches under development. Under the all branches tab on GitHub there are 7, yes 7, pages of branches! Tucked away in the middle of the sixth page, between countless Greenkeeper branches, I found a branch called 138-import-can-stache-bindings. This branch contained the fix I needed.

There may have been a smarter way to find the branch I needed, but relying on the GitHub web interface and following this procedure was the most natural way for me to go about it without thinking to hard.

Essentially, yes, the bug was the impetus. I would have been happy to not do any troubleshooting, but I haven’t yet successfully created a CanJS project using all of the newest features that runs flawlessly. My latest attempt was https://github.com/leoj3n/bitcentivez. Once the issues I’m having with it are resolved, it may be a good template for projects that want to use all the newest libraries and features of CanJS (such as algebra with Feathers v3 without having to strip out unused stuff from other more complex projects).

That’s cool with me, and I think that might look nice too.

Yes, they are explicit about this, but as long as you have Greenkeeper turned up to 11 it’s going to keep making branches and sending PRs for every patch and minor release. This isn’t necessarily a bad thing if the noise is reduced by accepting those PRs the day/week they come in (and deleting the branch).

In addition to decluttering, when a maintainer accepts the various Greenkeeper PRs in a timely manner, that says to noobs like me that the minor/patch release should be working with that package that depends on it (and shows confidence that you are testing that release with all the packages that depend on it).

Otherwise, I can become skeptical of whether that minor/patch release is one I should be including in the mix when troubleshooting. Essentially, I’m just thinking that it might provide the added benefit of making troubleshooting easier for someone that doesn’t know what the heck they’re doing in CanJS! :stuck_out_tongue:

I didn’t mean to suggest restricting the wide range, but you may be right that updating the minor or patch range numbers restricts the “wide range” in a bad way, or maybe it introduces some new kind of noise. If so, then still consider the negative affects of the Greenkeeper noise; I think turning down Greenkeeper, if possible, would be helpful to noobs like me that are trying to navigate many unfamiliar repos. :+1:

9

confidence that you are testing that release with all the packages that depend on it.

We don’t immediately test all downstream changes upon a release. This is why we make “official” CanJS and DoneJS releases. Our hope is that the vast majority of users stick to these official releases, and only in particular circumstances do they need to have a custom version.

With respect to greenkeeper, we use it for early warning of downstream change. The official “sanction” is an official CanJS release. This is because only “canjs” releases are fully tested together in every browser.

1 Like
10

Maybe one thing that isn’t obvious, is that greenkeeper only really works on a “release”. So we have to “release” the package to actually see how downstream packages are affected.

Also, the open-source saucelabs account only allows 5 repos. So we are not able to test every package cross browser before release.

This is why having “canjs” releases as official releases makes the most pragmatic sense.

1 Like