I’ve been working at Jewelbots for the past year and a half, and my work has revolved around three main areas, our mobile companion app for the Jewelbot, the firmware for the Jewelbot, and the backend systems that span the two.
For the mobile app, I chose Ionic. There were multiple reasons behind this, not the least of which is that it’s far cheaper and easier to develop one app in JavaScript than it is two develop two native applications, and I have never developed either a native iOS application or an Android application.
So Ionic it was. I built out the underpinnings of the app, turned it over to our resident amazing designer (seriously, Vico is top notch), and began to work on the firmware.
And that was it, until I decided to come back to the app after 8 months of not touching it. I tried to run the app:
grunt serve
and immediately saw:
Error: cannot find module 'osenv'
great. So after a quick Googling revealed my nodeJS environment was screwed up (remember: I haven’t touched anything since January, except to install minor updates to Mac OSX (Yosemite), I decided to uninstall node and reinstall it using Homebrew using:
brew install node4-lts
I didn’t use node 5 because at the time we were avoiding it due to breaking changes with Node5 and Ionic.
So I decided to do what any software developer would do. Ininstall and reinstall. During the uninstall/reinstall a different way cycle, I also fell prey to this npm bug, that added a few hours onto my work.
So after Node was deleted and installed again (this time from their website, not homebrew), I cd’d into my mobile app folder and ran:
npm install
Since we’re using npm-shrinkwrap, I fully expected everything to work. After all, it’s a fresh installation of node, a fresh installation of npm, and everything is shrinkwrapped and tied to specific versions. Right?
Wrong.
If you were around for the great left-pad debacle, you’re probably aware that before March of 2016, npm had no process of dealing with modules that were unpublished. They were just gone, and you were in trouble if you depended on them. This was one of those cases, specifically with a module named “i”:
NOTE: 0.3.2 was accidentally unpublished from the server and npm doesn’t allow me to publish it back. Please upgrade to 0.3.3 (source)
Now even after dumping the dependency tree using npm ls (both global and local), I couldn’t find where i was being used, so I started a bisect approach, where I’d remove half the npm modules from the package.json and see if that caused it to work or fail.
After a little searching, I found out it is a dependency of a few things I used, but the one that turned out to be relevant was Ionic 1.3.
But I wasn’t using Ionic 1.3, I had Ionic 1.7.16 installed. I verified that with
ionic -v
So what was going on?
It turns out, the answer was in my tooling. I had installed the ionic-generator for yo, and the generator’s Gruntfile.js expected both ionic and cordova (and a few other packages) to be available in the local package folder. I could see this by opening up the Gruntfile and trying to see where it was looking for packages:
var cmd = path.resolve('./node_modules/cordova/bin', exec);
and
var script = path.resolve('./node_modules/ionic/bin/', 'ionic');
Of course, I had installed ionic locally to rid myself of this error (I didn’t understand the generator’s tooling at the time, and just wanted to get running), and later on as they diverged, it caused this issue for me.
So how do you fix it?
Well, I first thought that I should just tell grunt to look at the global locations for cordova and ionic, since their documentation states they should be installed globally. Turns out that’s not easy to do in a platform-independent way. I assume (coming from Python) that there are virtualenvs that make this whole process easier; but I can’t get a straight answer which one should be used or why.
After a little while of Googling, I found two ways to fix the issue. The first:
npm link <package>
causes npm to reference the global package as if it’s installed locally. It’s a symlink from the local install location to the global install location.
The second is to use resolve-up. It provides a way to use globs to search global install paths for a package.
This was all my fault. I did the following things wrong:
- I didn’t understand the generator tooling enough to realize installing the same package globally and locally would cause an issue later on.
- I didn’t realize that for node, a user running a command and a script running a command are executed in different contexts (even though I should have, right?) I may have the global ionic-cli when I type ionic, but my Gruntfile doesn’t; it doesn’t get the benefit of my PATH (why doesn’t it?)
- I also didn’t realize that if you want to be absolutely certain the ground doesn’t shift beneath you, you should not only not have the same package installed globally and locally, but you should make sure you’re either using npm-shrinkwrap, or have your package.json pinned to specific versions, and hope they’re never un-published.
As a developer, I know intrinsically that this is all my fault. As a user, I have to ask why we don’t hold our tools to the same standard we hold other software?
Why doesn’t node or npm know you have the same package installed globally and locally, and warn you about it? Why doesn’t it it say, “Hey, instead of installing this locally, why don’t I link you to it?” Or, why doesn’t node install with a virtualenvwrapper? Why isn’t that the default method for working with node modules? If there is guidance from npm that all packages should be installed locally, but guidance from makers of CLIs that they should be installed globally; why hasn’t a third-way been worked out? Or is npm that third way? Why don’t those CLIs (when creating a new project) automatically npm link themselves?
Most of all, why do beginners have to be experts on dependency management in order to get a project up and running?
I love the idea of npm, and I love the passion around the community. But there are certain things that should work out of the box, and dependencies are one of them.