At EngineerBetter we generally categorise cloud-native transformation challenges into four problem spaces:
- Self-Service Platforms
- Continuous Delivery
- Application Architecture
- Organisational Change
Through our experiences with enterprises, we began to arrange these problem spaces into a mental model formed of concentric rings. We’ve found it useful to structure our thinking like this when we talk to customers, because it helps highlight how platform adoption tends to reveal new problems and constraints, in an ever wider context.
The way most enterprises operate is wasting both the time and money that they are made from. By learning from cognitive psychology and neuroscience we can find better, more productive ways of working.
The commonly-accepted working practices of today are born of the industrial revolution and the post-war era of mass production. It’s no surprise that these methods are best suited to paperwork factories and not the creative endeavour of solving business problems with software.
It’s no secret that we love Concourse. Concourse is a new kind of continuous integration server that treats pipelines as a first class citizen. It uses declarative config and a minimal UI to ensure that your builds can always be repeated, and your CI server always recoverable.
Concourse is notoriously easy to get started with, but as soon as you want your team to use it in production you’ve previously had to learn BOSH (which is notoriously hard to get started with). Teams who just want great CI shouldn’t need to think about this, so we built a tool called Concourse-Up to get your cluster up and keep it running, using a single command.
EngineerBetter are teaming up with Resilient Scale to deliver the full three-day Cloud Foundry Certified Developer training course in the days before the Cloud Foundry Silicon Valley Summit 2017.
It’s $1,975 for all three days with lunch provided. The course is held at the Santa Clara Convention Center, which is conveniently the same location as CF Summit. You can enrol via Eventbrite.
It’ll be a packed few days, full of hands-on practical exercises, interactive walk-throughs, and surprisingly few slides. The course is programming-language agnostic, and students also get the pick the brains of two highly-experienced Cloud Foundry consultants - Steve Greenberg and I.
This is just the beginning of a process of continuous improvement, but already we have seen the magic that can happen when you start working in small batch sizes and begin to organise Enterprise IT teams around a ‘platform as a product’ mindset.
EngineerBetter recently helped a large financial organisation fully automate their Pivotal Cloud Foundry deployments, taking a manual process that took a whole team more than a week, and replacing it with an hands-off continuous depoyment pipeline that took mere hours, often overnight whilst the team slept. We didn’t just automate the deployment of their PCF: we automated the creation of their cloud infrastructure; deployment of upgrades; installation of security updates; and all with full testing.
Cloud Foundry enables continuous delivery for developers. EngineerBetter bring continuous delivery to platform operators and traditional IT teams. Here’s how.
A customer of ours recently had trouble with their Concourse instance which resulted in us raising a GitHub issue describing how containers may get orphaned and that users may experience the error message
insufficient subnets remaining in the pool.
This can mostly easily be remedied by performing
bosh recreate on the Worker VM, or if you’re not sensible enough to have deployed your Concourse with BOSH, by locating and killing all the orphaned container processes. The debugging journey was rather fun and exposed some of the innards of Concourse and Garden, so we’ve decided to share it.
Imagine you’re responsible for operating shared IT services in a global enterprise, providing a self-service platform to your internal developers. If you were to continuously deliver this platform, in exactly the same way as cloud native apps themselves, what would it mean to your teams and your business?
Firstly, try to imagine a shared IT platform in a global company where its suddenly economical to work in small batch sizes, allowing you to actually embrace change rather than resist it.
EngineerBetter is now a systems integration partner of Hazelcast, Inc., specialising in helping people leverage the Hazelcast in-memory data-grid alongside their Cloud Foundry deployments.
We’re the only Hazelcast SI partner who are also Cloud Foundry Foundation members, making EngineerBetter the go-to consultancy for integrating the two technologies
We’re very pleased to announce that EngineerBetter Ltd is now a silver member of the Cloud Foundry Foundation.
EngineerBetter is the only UK-native consultancy that is a member of the Cloud Foundry Foundation. We’re also independent, with no vested interest in selling you one distribution or another.
I’m giving a talk entitled Brain-Aligned Delivery at Spark The Change. The presentation is an aggregration of various cognitive psychology and neuroscience findings that apply to working practices, and this post serves as accompanying notes, with links to its many references.
In Part 1 of this tutorial we set up a new BOSH director from scratch on AWS. Now it’s time to deploy Concourse!
What will we do next?
- Create a deployment manifest for Concourse
- Upload stemcells and releases, then deploy using BOSH
- Install and get started with the fly cli
- Try a more complex deployment
The Cloud Foundry ecosystem has given rise to some amazing things. This series of posts is about two of them - BOSH and Concourse. Together with CF, they form an awesome triangle of tools that will change the way you think about how to get things done when delivering software.
BOSH is the distributed systems powerhouse that makes Cloud Foundry possible. It answers the impossible question: How do you continuously deliver not just an app, but an entire platform of distributed virtual infrastructure?
Concourse is a intuitive, cloud-native CI tool that has a powerful simplicity to it. Actually, I’d go so far as to say it’s more than just an automation tool, it’s a medium for artistic expression! Just have a look at some of the vast Pivotal pipelines (e.g. the Buildpacks team) and marvel at their aesthetic value.
Why does EngineerBetter exist?
We started this company to make things better for people who build software for a living. This includes both our customers and our employees.
EngineerBetter was founded in the belief that today’s status quo is not good enough for humans. We think there should be less friction in the way we work. We should be more effective and we should learn quicker.
I’m talking at Cloud Foundry Summit Berlin 2015 - please do feel free to stop me for a chat!
Between versions 206 and 211 of Cloud Foundry a bug causes
buildpack_cache files to be orphaned, increasing persistent disk usage unless manual remedial action is taken.
Users will get the error message
Staging error: cannot get instances since staging failed when trying to push or restage apps. Depending on your setup you could get higher AWS S3 bills, or you may run out of persistent disk on your Cloud Controller or NFS Server VMs.
If you play the dangerous game of scaling Cloud Foundry DEAs up instead of out, make sure you have plenty of headroom in the CPU department to start with. If you don’t then there’s a good chance you’ll see a heap of
no available stagers errors upon
cf push and
insufficient resources to start errors upon
cf start. This will last for an hour before magically fixing itself.
Why is this? What causes all these errors, and what can be done if one gets it wrong?
I’m back from the excellent Cloud Foundry Summit 2015, and have been back in the country long enough to catch up on some sleep. That picture up there is the poolside area where I may have sipped a few pints of Californian IPA between presentations.
There were plenty of interesting talks about specific subjects (CC V3 API, Service Broker API updates), but I wanted to recount some common topics that cropped up.