Windows to Linux Development Part III: Reviewing Linux Setup

In the previous two posts, I’ve compared and contrasted Windows vs. Linux, .NET vs. Python; but I haven’t dived (dove? diven? English is so confusing(!))  into what development actually looks like on Linux. For you .NET devs, I’ll try to draw parallels as I go.

 

I use Ubuntu as my operating system, but after trying (unsuccessfully) to use rdesktop, I moved over to Xubuntu.  I chose Ubuntu because of its driver support and ease of set-up compared to other Linux distros I’ve tried in the past.  

Python development (specifically covering the Django framework) needs a few tools put together in order to work correctly:

virtualenv: a tool that creates a separate virtual environment for storing packages used in a Django project.  In .NET, you can place libraries (packages) in the GAC, or you can use NuGet to install them into the project and bin deploy them along with the project.  In Python, by installing a package into the virtualenv, you’ve essentially done the same thing a bin deploy would do in .NET.  If you don’t install packages into a virtualenv, then you’re essentially GACing the libraries.  This is a bad thing for the same reason it’s a bad thing in .NET: You can’t have two DLLs (packages) with the same Assembly name.

virtualenvwrapper: A tool that makes virtualenvs easier to set up and maintain. instead of this command to load a virtualenv:

source ~/.venv/myproject/bin/activate

it’s this:

workon myproject

Nginx: nginx is a very versatile web server.  It can be used to reverse proxy calls such that instead of the following:

http://0.0.0.0:8000/

you have:

http://myproject.myboxname.com/

IIS provides this capability as well.

uWSGI: A lightweight web server that requires zero set-up; it’s used as a development web server for Django projects.  Its .NET equivalent would be the Cassini web engine.

byobu: an enhancement for tmux that allows a user to have multiple terminal (command line) windows open at once, and even have splits in the same window. It doesn’t have any direct equivalents on the Windows side, except to say that Console2 tries to take care of the tabbing functionality.

apt-get: A system package manager for Ubuntu (and other debian based systems).  It allows you to install new software with one command.  I heard they’re bringing it to Windows, and all I can say is, what took them so long?

pip: A python package manager.  Nuget for Python.

PyCharm: A Python/Django IDE by JetBrains, the makers of ReSharper.  It literally feels like Visual Studio for Linux. Sadly, it runs on Java, but those are the breaks.

Vim: I can’t do Vim justice. It has a steep learning curve and I have yet to scratch the surface of what it can do for editing code.  I still haven’t decided on whether I’ll stick with Vim or move to Emacs.  Vim’s modal nature is my biggest complaint about it. 

I ended up moving to Vim from PyCharm after my Ubuntu system crapped out on me when I made the mistake of trying to uninstall a beta version of Crossover Linux.  After a fresh system install (luckly my Home directory was on a separate partition) I decided to forgo PyCharm and stick to Vim.

 

Problems with using Linux for Development

Installing updates is problematic, to say the least.  The Ubuntu software updater doesn’t separate out kernel updates from other updates, and if you’re not paying attention, you could inadvertently try to update the kernel while updating a package. Maybe it’s just my bad luck, but this has always resulted in issues for me that were not trivial to figure out (for a linux newbie such as myself).  

Window desktop manager crashing is far more common than it should be. Linux is really stable, Window Desktop managers are not. Here are a list of things that have crashed my window desktop manager, in no particular order: Starting Pidgen, opening an IM window, hitting ‘play’ on a piece of audio. Opening Chromium,  booting up, or installing updates. I’ve gone from the proprietary Nvidia drivers to the VESA drivers, to whatever I can try, and it still happens.  It doesn’t happen often, but when it happens it’s really annoying.

There is no good support for corporate (synonymous with Exchange and Outlook) email. After a lot of effort, Thunderbird can be used to read and write email, but if you’re using Outlook’s calendar, you’re pretty much stuck. If you use anything more advanced than Outlook’s Email and Calendar (like Tasks, To-Dos, or processing rules) you’re really stuck.  Your options become:

1. Try Crossover

2. Set up a terminal services login to a box that runs Outlook, and using rdesktop to connect to it.

3. Use Outlook Web Access, in all its (ahem) splendor.  The problem with this approach (besides the fact that it’s OWA) is that you can’t process existing rules in OWA. 

Crossover simply does not work for Outlook 365 (or any of the 365 services, come to think of it). A Terminal Services login is an ok alternative, but it has its own issues (copying/pasting, opening links, etc).

Corporate IM / VOIP service is almost non-existent.  We run a Cisco VOIP system, and they do not have a Linux client.  Even worse is that if you have the Mac version of their software, it can only talk to other Macs — it can’t do voice communication with Windows VOIP users. I wish I were kidding.

Overall, the switch from Windows / .NET Development to Linux / Python development has been a great learning experience, but looking back, I’ve spent too much time configuring and getting everything to work with everything else, and that should not be the way it is.  It’s 2013, we can’t do this better?

In the final part, I’ll detail what I consider to be the most optimal solution for a python developer to do both Windows and Linux development without Dual booting.

 

Advertisements

From Windows Development to Linux Development: Part 2: .NET to Python

It was December 11th, 2012, and I was scared. Not because of the impending Mayan apocalypse, mind you, but because it was my first day doing something completely alien to me: developing software on a new platform, a new language, and having a completely new product to deliver.

That’s a lot of change to handle at once.

Looking back, .NET development would probably be just as scary to a seasoned Python developer as it is the other way around. At least I hope so, because Python development was initially very scary to me.

.NET development is based (almost) singularly around Microsoft’s tooling, and Microsoft has deep pockets, so they can spend money on marketing and great documentation. Where their documentation fails (and it does fail — who wants to study method signatures all the time?) the MVP community takes over and writes eloquent blog posts about some esoteric part of a new framework that behaves quite unexpectedly.

There are certainly not a lot of popular web frameworks in .NET. If you’re in the corporate world (hello, 80% of you), you’re likely either on Webforms or ASP.NET MVC. That’s a really small playground comparatively, making it much easier to ‘ramp up’ (especially if you include the sheer number of questions on Stack Overflow regarding those frameworks.

In Python, you’ve got a number of smaller frameworks, with Django as the BMOC. Django’s documentation is a complete 180 from MSDN, with an emphasis on prose over method signatures.

With .NET, the answer is normally a code sample away. With Django, it could be a code sample away, but normally it isn’t, it’s buried in some prose. An example that comes to mind is formsets, which were (unluckily for me) one of the first things I had to deal with in Django.

With Python development, you necessarily have access to the source code. For instance, you can read through the source code for Django on github, or if you have a neat editor like PyCharm, you can simply click to view it (much in the same way ‘Go to definition’ works in Visual Studio).

For Windows based development, the toolchain normally looks like the following:

  • Visual Studio 2010/2012
  • TortoiseHG (or Git for Windows)
  • ReSharper (a must have. Really.)
  • KDiff3
  • IIS
  • SQL Server Management Studio (regardless of DB location)
  • Terminal Services (Remote Desktop)
  • A really beefy machine. Really Beefy:
    • 8-core Intel i7 @ 2.93 Ghz
    • SSD (in my case, a 120 GB)
    • 8 GB Ram
    • Nvidia 9600 (For dual monitors)

For Linux based development, (at least in Python):

  • Pycharm or Vim (Or Emacs, but do we really need to have that fight here?)
  • TortoiseHG (still a plus — though I can’t get it to run on my current system)
  • KDiff3
  • Nginx (used as a proxy)
  • uWSGI (web server)
  • No DB installation required (normally Sqlite), or PostgreSql or MySQL (All three are used in my project)
  • Same beefy machine still works, but entirely possible to do all work on a remote linux server with very little fuss

    Almost the first thing I noticed when starting out with Python development is just how easy it is to debug. In .NET, the workflow is:

    • write code
    • run unit tests (you do have those, right? RIGHT?)
    • compile
    • wait for application to JIT on IIS (or reset IIS and then wait for application to JIT if you’re using Spring.NET or some other DI/IoC that hangs out in resident memory)
    • find error
    • start all over again

    Even on the beefiest of machines, that entire loop is still at least two minutes (I’ve seen it take as long as 5 minutes on larger solutions). That’s a long time to be staring at a screen.

    In Python (specifically developing in Django), here’s the workflow:

    • write code
    • save code
    • refresh website
    • find error.

    Time? About 10 seconds. Maybe. That’s an incredibly fast turn-around time, and makes unit testing less attractive (that’s not the only thing, which I’ll get to later).

    What makes the workflow in Python so spectacularly different? It’s the nature of the beast. While Python isn’t strictly interpreted , it still doesn’t have the overhead compiling a .NET assembly has.

    Another difference in Python development is just how easy it is to write scratch code (I’m thinking specifically of Django’s “shell_plus” module, but the same holds true with iPython):

    It’s literally one command:

    python manage.py shell_plus

    And that command drops you into a REPL that has all the items in your Django project imported. Need to load a model? It’s already there. This somewhat reduces the need to use unit-tests as scratch code. You can test behavior in the shell, and then add it to your program.

    Database development in Django is completely different than in any .NET project I’ve ever worked on. Django has its own built-in ORM, and the mere act of creating a model creates a Database table. With the South migration library, you can even version control your database and it’s all in one place. It’s simple to set up, and compared to the .NET model of database development, it’s painless. But it’s not without its own problems:

    – The default is to have your Model objects be Django DB Model objects, and that means they’re bloated, ugly, and carrying around persistence information. This complicates unit testing (and unit testing is no longer unit testing when you bring in a database). If you want to get around that, maybe with a Functional Core, Imperative Shell model, you have to do *a lot* of work. I’m currently trying this approach on a personal project, and it *sucks*.

    – Unit testing a Django application is not unit testing. By default, it pulls in your entire database. That’s integration testing. SSDs are required if you want your ‘unit tests’ to be fast, because Django is happily spinning up a database for your tests.

    In .NET, database development reminds me of building a house from scratch *every*, *single* time. There have been improvements with the Code First Entity Framework stuff, but nothing as turnkey as Django’s ORM.

    Deployment wise, .NET deployed applications don’t need as much scaling out of the box for large sites as Python/Django projects do. The deployment tooling for python projects isn’t as great as some of the solutions for .NET (I’m thinking of Jenkins vs. Octopus for .NET). I’ll get into deployment tooling for Python in a later post, but essentially it’s built for developers to deploy, not for *anyone* to deploy.

    Overall, I’d have to say that .NET and Python development have different strengths. Neither side has a clear advantage over the other, at least for *large* projects.

    In Part III, I’ll talk about my Development set-up, and why I’m moving to Mac OS X for Python development.

  • From Windows development to Linux development

    About 6 months ago, I was given the opportunity to start working in Python and Django. I’m always excited to learn new things, so I jumped at the chance.

    Since our Python projects are deployed on Linux servers, my choices were to:

    – Stay on Windows and use Vagrant
    – Dual boot into Linux and do Python development there

    I chose the latter, for multiple reasons:

    – Gave me a chance to improve my linux-fu
    – Closer to what is actually run in a production environment
    – No magic. Setting up everything myself gives me a chance to ‘go through the pain’

    8 months later, I’m happy to say I made the right choice*.

    Some things I first noticed when working in Linux:

    Python (and by extension, Linux) doesn’t have the DLL hell that we encounter on Windows. They have their own special hell, and have created software to work around it (virtualenv and its much needed companion, virtualenvwrapper)

    Pypi and PyPy are two separate things.

    Working in Linux is a whole lot less about polished GUI software and a whole lot more about using the neat little utilities the operating system provides together to produce the same outcome. It’s the difference between having a ready-made lego deathstar, and building your own.

    Linux machines are not more stable than Windows machines. They are not less stable than Windows machines. They are differently stable. Windows machines (at least now) have good tooling around errors (Event Viewer) and a standard that says where errors should go (think of it as a hierarchy in your application), whereas with linux machines, it could be in a few places. /var/opt/log or /var/log/ or somewhere else entirely. When a sound device crashes in Windows, it doesn’t normally take down the whole desktop any more; but in Xubuntu 13.04, launching a media player will take down my entire desktop manager (xfce); as will launching Pidgin (but only sometimes).

    That having been said, I can easily update Xubuntu without restarting my PC, where any little change on my Windows machine requires 5 reboots.

    The year of the Linux desktop isn’t here yet. I’ve run a few Desktop Window managers, and I have yet to find one that is stable enough to withstand day-to-day use.

    Things I’ve noticed:

    Crossover does not work with Outlook 365. Still. I’m a beta tester for their products under the hopes it may one day work for Ubuntu, but no dice as of yet. It works just fine for ‘normal’ Outlook, but once you include the Cloud, things get crazy.

    Vim is mode driven. Depending on whether you’re in Visual mode or Insert mode, stuff works differently, and you have to constantly look down to see what mode you’re in.

    Byobu and the mercurial_keyring extension do not always play nicely together. I’ve left a session on, gone home, come back the next day, and tried to pull only to have it not work with a gnomeKeyring.IO error.

    The command line isn’t as much of a pain in the ass as I thought it was when I used Windows daily. The tooling around the command line (I use Bash) is such that once you get the hang of it, it enables you to work more quickly. I think Microsoft would do us all a favor if they just adopted cygwin as a replacement for their command shell.

    X-Forwarding (the act of working with a graphical window from another PC on your PC) is problematic to set up. I tried for an entire day (when I probably should have been working) to set it up, and never got it to work. I eventually set up a dual boot Windows 8 / Ubuntu setup on my home PC (which I later regretted).

    VNC is a terrible alternative to what Windows users have built in (Remote Desktop). If I remote into a computer with VNC, that computer is ‘unlocked’. Anyone can sit down and start using it while I’m remoted into it. That’s a Bad Thing in an office where co-workers like to pants each other.

    Printing is hit or miss. Nothing will have changed, it just doesn’t work sometimes. I will say that in the case of the corporate network copier/printers we have in the office, it mostly worked the first time.

    Finding answers to problems in a linux system is 80% some ill-begotten forum or mailing list, and 20% AskUbuntu. I wish more results were on Ask Ubuntu.

    Playing video or audio is now a struggle. I had to download 3 different programs just to find one that would play a .WAV file from my email. I’m sure there was just some dependency conflict somewhere, but I’ll be damned if I knew how to find it.

    To sum it all up: Those things that you’ve expected to ‘just work’ on a Windows box no longer ‘just work’ on a Linux box. In return for this pain, you get (what turns out to be) a very cool platform for software development, as I’ll go into in Part 2.