Zealots repellent

In this article I am going to take some liberty and refer to FLOSS (Free and Libre Open Source Software) as « open source » or « free software ».

Teh origins

A few months ago I was put in charge of developing an open source ${THING} at D2SI. THING was still undefined at the time but could have probably been found at the intersection of strategy, cell, offer, communication and vision. At first I was amazed that in 2017 companies still needed to be introduced to the FLOSS ecosystem. Anyway I thought it was an interesting idea and got very excited, very fast!

Oh boy, little I knew I’d be left alone naked in front of that task…

After a few days of thinking, I came to the conclusion that I had no clue what I was doing. I mean I’ve been contributing to and using open source software for about two decades but what the hell could a consulting company possibly offer to the community? We are not a software company. So what does it mean for us to have an open source ${THING}?

Sure we are using FLOSS on a daily basis but again, we’re not a development company, so it’s not like we have code we can open source… or have we? We spend countless hours everyday writing IaC (infrastructure as code). Could that be something worth sharing?

Fr33 7H3 C0D3

This almost immediately brought up the question of added value. As a consulting company, what do we sell? When I look at people spending their days on stack overflow and copy/pasting random things from the internet, I am convinced that our added value is not in our terraform recipes for example but in our experience and reasoning that led us to build things the way we do.

We don’t believe there is any negative impact in opening our code, recipes and development process. Plenty of individuals and organizations already do so. It’s not trivial to incorporate outside code and in general it’s much more beneficial for our customers to come to us to do the integration together because that’s where our expertise kicks in.

But let’s not kid anyone here, that approach is also highly beneficial to us. Because working in all kind of different setups will help our consultants to enhance the code and come up with something reusable and multipurpose (i.e. not tight to a particular environment). The goal is to spend more time on things that matter and are fun.

So by itself our IaC has no great interest. But shared amongst our pool of consultants and the community at large, there is a chance we can make it awesome for everyone…

OK, that could actually work. Let’s take it for a test drive! Of course IaC is just a poor excuse to kickstart the project but we have to begin somewhere. Talking to people around D2SI I came to realize that there are many other great things we can share with the community as we will see below.

Here be dragons!

Then there was the question of the license. What should we use by default in our projects? As most people familiar with FLOSS know, discussing this topic almost always ends up in a religious war.

Now, I’m a BSD guy and I don’t like viral copyleft licenses like the GPL. I much prefer the « no strings attached » approach of BSD-style licenses which basically says:

• do whatever you want with the code, yeah that includes building a nuke
• don’t complain if it does not work or kill your puppy
• don’t pretend you wrote the code (no one would believe you anyway)

<troll>Yes I am aware I am just throwing a totally subjective opinion up in the air.</troll>

Anyway, because I am old and wise, it wasn’t hard to convince management to go for one of the most permissive licenses: the ISC license which is used by the Internet System Consortium (duh!) and most importantly OpenBSD.

Side note: if one ever wanted to understand some of the differences between open source and free software, looking at the 3.3 clause of the Amazon Software License would be a good start. Code distributed under this license may be open source but it’s not free software (usage limitation).

EPIC FAIL: GitHub is not free!

Yeah yeah yeah, GitHub is not free software, but like or not it won… for now. So let’s be pragmatic and put things where people expect them to be. Also everyone knows how to open a pull request and contribute to a GitHub project. We are trying to make it easy for anyone to do so or report issues.

As of today, here are a few projects hosted at https://github.com/d2si-oss :

• data-engineering-coding-challenge
  Coding challenge for future D2SI data engineers
  This is a technical test for data engineering candidates. They should do this test on their own. It’s designed to take 2 or 3 hours but there is no hard limit.
• demo-aws-lambda-buffer-api
  Deploy a serverless API Buffer using AWS Lambda and API Gateway with Terraform
  The basic requirements for the project are an HTTP endpoint that accepts a JSON GET/POST/PUT/DELETE/OPTIONS containing random data which will need to be stored somewhere. This required a few components:
  – a front-end to take in HTTP requests: API Gateway
  – a back-end for handling these requests: Lambda
  – a datastore to keep all the associated short tokens and API responses: DynamoDB
• ooso
  Java library for running Serverless MapReduce jobs
  Ooso lets you run MapReduce jobs in a serverless way. It is based on managed cloud services: Amazon S3 and AWS Lambda and is mainly an alternative to standard ad-hoc querying and batch processing tools such as Hadoop and Spark.
• terraform-modules
  Reusable Terraform modules
  This repository holds modules for the terraform IaC tool. They are arranged by providers type (e.g. aws, google…). The root of the repository contains sample stacks exercising some of these modules.

This last repository is still pretty young and small because it is meant to onboard D2SI consultants. You catch more consultants with honeyform than you do with CloudVinegar.

We also encourage our teammates to fork upstream projects they want to contribute to under the d2si-oss GitHub organization. That’s how we ended up contributing to a few emblematic projects like terraform or the aws-cli.

We’re just starting but we already have a few projects in mind like a training environment bootstrap recipe using packer, terraform and/or ansible for instance. We offer different kind of trainings via Icelab and there’s no question people would appreciate a bootstrapping kit to automate student requirements (instances, user account, pre-installed software etc.) by just defining a few variables.

We would also like to promote some technologies that we believe are better suited for some tasks than the industry would usually promote. OpenBSD being an example (multiple region / cloud providers connectivity over IPsec, SSH bastion, RDP proxy, DNS forwarder…).

Not everything has been published yet because we are still cleaning legacy code, anonymizing to make things are generic as possible (e.g. Ansible roles). Also some of our internal projects are still poorly documented and we consider bad documentation the same as bad code so we need to fix that before opening it.

SAMCRO

We realized eventually that building a team around a project like this one made perfect sense for a consulting company. Consultants spend most of their time at customers so it’s tricky to build a team spirit and a sense of belonging. Sure we’ve been doing loads of internal events so that people could meet, exchange and share but that is different. Working together on such a project really makes you feel like you’re part of the club and gives you a different sense of purpose that of your daily job. We like to see it as a job spin-off. Gathering people around a common goal matches the way most free software projects are developed with contributors spread around the globe.

I feel like I need to stress something now: this proposal is solely based on volunteering, no one is forced to join the effort. Also this is not a way for your employer to coerce you into doing extra work, actually quite the opposite: while chatting with fellow co-workers it was brought to my attention that more than a few of them already contributed in their own time on software we use in our daily job. We can make this a win-win by giving them time during work hours to contribute: they do what they like, it makes us look good and we become more attractive to new recruits. We could even offer fun rewards to our top contributors!

There are also obvious benefits to everyone: a few weeks ago we had the choice between implementing something ugly or extending terraform with a new feature to satisfy a specific deployment need. We made time for our terraform contributor to develop the feature which made our customer very happy. It’s been merged upstream since, having to maintain a local fork and a local build would make no sense whatsoever.

Some people also expressed the will to contribute to upstream projects but need help to do so. Not technical but emotional help because it’s not always easy for anyone to confront ideas, deal with rejection etc. We think we can accompany them in that regard the same way we encourage our staff to speak at meetups and conferences.

We are also looking into hosting hackathons to create reference implementations of recurring infrastructure tasks, agree upon our best practices and stop reinventing the wheel each time we start a new project.

Shut Up and Hack!

If it weren’t for open source and the free software movement, we would all be working with immensely boring closed source technologies. Well not me because I’d probably be doing something else… Not to say our daily work isn’t surrounded with closed and opaque black boxes (I’m looking at you Cloud providers) but as least most of the tooling, software and operating systems we use are open source and more importantly free (as in freedom).

Anyway we are trying to build our own ${THING}, learning as we go to ultimately have fun! It will take time but we are confident we can create something that will work out for all of us at D2SI. We are not afraid to show what we do, what we know what we don’t know, how we work… And we sometime even talk about it in our brand new technical blog: http://techblog.d2-si.eu/.

An open source project is all about the community. Without it, it’s just yet another code rot repository. While we hope to create a community within D2SI to begin with, it is our goal to broaden the scope. So have your pick of https://github.com/d2si-oss. We value external feedback as much as internal one and there’s no precedence whether you’re a consultant at D2SI or not: https://github.com/d2si-oss/contributing-guidelines.

After working towards corporate liberation to unleash the potential and initiative of our staff, liberating the code was a natural next step.