Jenkins X at DevOps World | JenkinsWorld San Francisco 16-19 Sep 2018

Jenkins World has combined with DevOps world to become the premier DevOps event in your calendar this year - and there really isn't much time left if you still have to register.  You can even use my discount code to get even more value out your visit - its JWRDAVIES.

CI/CD on Kubernetes is really hot - which is why conveniently Jenkins X is going to be so prominent this year.





There are talks on Jenkins X:

Jenkins X: Continuous Delivery for Kubernetes

Extending Jenkins X for fun



There are workshops on Jenkins X:

Building Continuous Delivery for Microservices with Jenkins X


There will be community booth sessions on Jenkins X and there may even be the odd product announcement in a keynote thing that may or may not mention Jenkins X and there could even be a book about Jenkins X :)

So if you are in the middle of transitioning to the cloud or even just thinking about it -  then CI/CD is really an integral part of that transition and the best, cloud-native way to do that effectively is with Jenkins X - you don't even need to learn anything about containers or Kubernetes to start using it. So if you want to be a cloud-native rockstar genius this is one conference you can't miss (well, you can always attend the one in Nice, France in October).

Making Cloud Native simple: Jenkins X

You have probably heard that all companies are now software companies and that to compete you need to be embracing cloud native applications to ensure your company has the ability to adapt quickly - or you will soon be out of business. This is true by the way, but its also a little daunting for all of us. You have to be an expert in your programming languages of choice, be well versed in how your business works, and now need to understand what exactly cloud native means, what are containers and what are the latest trends in cloud technologies.

So lets make a quick list of things you should be doing:

1. Decompose your monolithic applications - or not. To generalise, you probably really should give this a lot of careful thought: its domain and application specific - i.e. only you can decide if it make sense (but it probably does).

2. Containers - or Docker or Moby - which really you need to think about containerd, or rkt  or something that complies with the OCI. Which is why people talk about containers - its all got too confusing.

3. Orchestrating your containers - i.e. service discovery, auto scaling, fault tolerant platform to run your containerised applications. Thankfully, this recently got a whole lot easier - with Kubernetes emerging as the one that will be supported by all the major public clouds, and existing cloud platforms.

4. Configure, release version, rollback and inspect your application deployments in a standard, or de facto standard way. Luckily Helm is excellent for this.
5. CI/CD - in order the extract the most value from cloud native, you need a continuous delivery system that will enable a predictable, repeatable release process, and enable continuous improvement via streamlining your development processes.
6. Monitoring - ideally you want to monitor the performance of your deployed applications and feed this back into your CD system.

This is an intimidating list to look at, but its probably already out of date because technology is moving at such a pace its really difficult to keep up.

What if you could abstract yourself away from all the technology bleeding edge and save yourself the paper cuts - and just concentrate on adding business value? Well this is the aim of Jenkins X - a new project that is part of the Jenkins ecosystem, as explained by James Strachan here.


We started Jenkins X at the begining of the year, taking the experience gained from the fabric8 project to develop an open source system that was targeted at these aims.  By concentrating on Kubernetes and utilising its wider ecosystem at the Cloud Native Computing Foundation, we have been able to develop a robust, targeted project that focuses on the needs of developers of cloud native apps.

In summary Jenkins X provides the following:

1. Abstracts away the gnarly bits of cloud native (you probably don't want be concerned with using skaffold for example). Its all still there, you can peel back the curtain as much as you like, but its nicer if you don't have to.
2. Automated CI/CD Pipelines - using Jenkins, configured to work well for cloud native
3. Environments - promotion using git based work flows, and preview environments  for pull requests.

This is all setup for you - you can aim Jenkins X at an existing project (or create a new one from scratch) - and select your cloud provider of choice and let us do the rest.

A small(ish) caveat: we target Kubernetes, so check here first.

Most public cloud providers allow free trials, or provide credits - which is more than enough to kick the tyres with Jenkins X - or you could try a local-machine solutions, such as Minikube.

Red Hat launches openshift.io

Red Hat launched openshift.io at Red Hat Summit 2017.

Its a SaaS based development environment, that provides everything you need to get up and running and start producing code.  The marketing speak actually says is a "A free, end to end, cloud native development environment".

This obviously caused a lot of questions for developers - it's sometimes (well, 99.99%) difficult to join the dots between marketing and reality. This caused some questions on Hacker News, but luckily Tyler Jewell, CEO of Codeny, was on hand to provide more clarity - its worth a read.

Just to reiterate - what openshift.io is going to provide for developers is a really  convient  hosteddevelopment environment to start their journey to developing apps for the cloud.

1. A deployment environment for your code native apps: You can use GitHub as your src repo, but deploy onto openshift to run your apps, for free (within limits). Red Hat wants to give developers the opportunity to run reasonable applications at no cost, but they don't have bottomless pockets, so there has to be some reasonable constraints - obviously you can chip in if you need more space/cpu.

2. A continuous deployment pipeline, that will enable you to run any code changes through test, stage to run, (which uses fabric8 and openshift pipelines under the covers).

3. A web based IDE, based on Eclipse Che, so you can edit your code in situ, without leaving openshift.io. This is a great feature for development, and allows you to develop and test the application you are building. However, if you don't want to leave the comfort blanket of your favourite IDE running on your laptop, and just push code to GitHub for openshift.io to pick up and deploy, that's OK too.

4. Analytics built in (using fabric8 analytics): to identify security risks in dependencies you maybe using, and also identify other dependencies that might be a better fit (e.g. flag you're using a really old version of commons math).

5. Agile management, to allow you to plan, and track development items for your code. This is really useful for collaborative development.

The fabric8 ecosystem also provides lots of developer tooling and examples to help developers get started.

Fabric8 keeps on growing

The fabric8 project has been at the forefront on innovation to enable software professionals to develop and deploy faster to cloud native environments. With the announcement of openshift.io at Red Hat Summit, 2017, the eco system of fabric8 has expanded to incorporate all (except the IDE - which is based on Che) of the technologies that make up the openshift.io developer platform:

The fabric8 project will continue to innovate and increase its scope to ensure it becomes the best environment for developers to continue to accelerate from idea to production. The fabric8 platform consists of over a hundred repositories under the GitHub fabric8io organization - all built on top of the fabric8 platform itself, keeping all releases in sync, and allowing us to continually improve our delivery.

Microservices Journey with Apache Camel

Moving anything towards a container based, Microservices architecture is pretty hard. Doing it while the technology has been evolving so quickly is a even harder, but creating a user experience that means Microservices is really easy to build is the hardest thing we’ve ever done — and we would love to tell you about it!

If you are in Atlanta or Minneapolis the first week of October, Red Hat is hosting A Microservices journey with Apache Camel, where James Strachan, Claus Ibsen, Christian Posta, James Rawling and myself will tell you about the expedition through containers, kubernetes/OpenShift, continuous delivery, API management and how we’ve made it all super easy to use with fabric8!

Fabric8 version 2.0 released - Next Generation Platform for managing and deploying enterprise services anywhere

The Fabric8 open source project started 5 years ago, as a private project aimed at making large
deployments of Apache Camel, CXF, ActiveMQ etc easy to deploy and manage.

At the time of its inception, we looked at lots of existing open source solutions that we could leverage to provide the flexible framework that we knew our users would require. Unfortunately, at that time nothing was a good fit, so we rolled our own - with core concepts based around:

• Centralised Control
• runtime registry of services and containers
• Managed Hybrid deployments from laptop, to open hybrid (e.g. OpenShift)

All services were deployed into a Apache Karaf runtime, which allowed for dynamic updates of running services. The modularisation using OSGi had some distinct advantages around the dynamic deployment of new services, and container service discovery, and a consistent way of administration. However, this also meant that Fabric8 was very much tied to the Karaf runtime, and forced anyone using Fabric8 and Camel to use OSGi too.

We are now entering a sea-change for immutable infrastructure, microservices and open standardisation around how this is done. Docker and Kubernetes are central to that change, and are being backed with big investments. Kubernetes in particular, being based on the insurmountable experience that google brings to clustering containers at scale, will drive standardisation across the way containers are deployed and managed. It would be irresponsible for Fabric8 not to embrace this change, but to do it in a way that makes it easy for Fabric8 1.x users to migrate. By taking this path, we are ensuring that Fabric8 users will be able to benefit from the rapidly growing ecosystem of vendors and projects that are providing applications and tooling around Docker, but also frees Fabric8 users to be able to move their deployments to any of the growing list of platforms that support Kubernetes.  However, we are also aware that there are many reasons users have to want to use a platform that is 100% Java - so we support that too!

The goal of Fabric8 v2 is to utilise open source, and open standards. To enable the same way of configuring and monitoring services as Fabric8 1.x, but to do it for any Java based service, on any operating system. We also want to future proof the way users work, which is way adopting Kubernetes is so important: you will be able to leverage this style of deployment anywhere.
Fabric8 v2 is already better tested, more nimble and more scalable than any previous version we've released, and as Fabric8 will also be adopted as a core service in OpenShift 3, it will hardened at large scale very quickly.

So some common questions:

Does this mean that Fabric8 no longer supports Karaf ?
No - Karaf is one of the many container options we support in Fabric8. You can still deploy your apps in the same way as Fabric8 v1, its just that Fabric8 v2 will scale so much better :).

Is ZooKeeper no longer supported ?
In Fabric8 v1 - ZooKeeper was used to implement the service registry. This is being replaced by Kubernetes. Fabric8 will still run with Zookeeper however, to enable cluster coordination, such as master-slave elections for messaging systems or databases.

I've invested a lot of effort in Fabric8 v1 - does all this get thrown away ?
Absolutely not. Its will be straightforward to migrate to Fabric8 v2.

When should I look to move off Fabric8 v1 ?
As soon as possible. There's a marked improvement in features, scalability and manageability.

We don't want to use Docker - can we still use Fabric8 v2?
Yes - Fabric8 v2 also has a pure Java implementation, where it can still run "java containers"

Our platforms don't support Go - does that preclude us from running Fabric8 v2 ?
No -  although Kubernetes relies on the Go programming language, we understand that won't be an option for some folks, which is why fabric8 has an optional Java implementation. That way you can still use the same framework and tooling, but it leaves open the option to simply change the implementation at a later date if you require the performance, application density and scalability that running Kubernetes on something like Red Hat's OpenShift  or Google's Cloud Platform can give you.

We are also extending the services that we supply with fabric8, from metric collection, alerting, auto-scaling, application performance monitoring and other goodies:

Over the next few weeks, the fabric8 community will be extending the quick starts to demonstrate how easy it is to run micro services, as well application containers in Fabric8. You can run Fabric8 on your laptop (using 100% Java if you wish), or your in-house bare metal (again running 100% Java if you wish) or to any PaaS running Kubernetes.

Fuse At the DevNation Conference!

The JBoss Fuse engineering team have sponsored and organised CamelOne for the last 3 years, but after CamelOne 2013, the opportunity came up to put all the effort into a new developer conference, sponsored by Red Hat called DevNation. This is the first time the event has been run, and its a great opportunity to learn about all aspects of development and deployment. CamelOne was focused on Apache projects used for integration, but that in itself is quiet limited, and as an integration developer, you have to be able know about so much more. DevNation is an opportunity to learn from like minded developers about all aspects of real world deployments, from Hadoop to elastic search, from best practices in DevOps or OSGi,  to getting an insight into Docker, Apache Spark, Elastic Search and so much more. DevNation has a lot of promise to be a great developer conference, with a broad scope that will be informative and fun. Its for this reason that the fuse team decided to focus our attention on DevNation this year, rather than CamelOne.

The traditional way of delivering applications is outdated. Many users are rolling out across hybridised environments, and the need to be insulated from all the different environments,  to have location independence and the ability to dynamically deploy, find and manage all your integration services is going to be the key theme for the Fuse tracks at DevNation - as well as all the usual tips, tricks and secret ninja (OK undocumented) stuff that we like to share with the attendees.

DevNation this year is being held in San Francisco, and will run from Sunday April 13 - 17. You can register here - and we really hope to see you there!

One Technology Trend for 2014: "The Internet Of Things"

I was reading some online articles and came across Technology Trends for 2014 - the number one being the 'Internet of Things' or IoT for short. This isn't exactly a new concept, - the promise of smart homes, where everything from intelligent lights to A.I. for washing machines that can be monitored remotely have been around for a while. And how couldn't resist the concept of a smart fridge that can stock itself? The term Internet of Things has been around for over decade, being firstly proposed by Kevin Ashton whilst Auto-ID Center at MIT, primarily driven by an interest in RFID, but the ideas and uses cases for an Internet of Everything has taken a while to mature.

The drivers behind the IoT have been several:  The demand for renewable energy means that smart grids have to be able to monitor and respond to demand in electricity generation in a more agile manner - allowing for bi-directional energy supply from small energy producers (potentially you and me) - requires smart metering and monitoring. There's the exponential growth of smart phones  - more people are always connected, and that trend will continue.

However, when the IoT was first envisaged all those years ago, there were some technology inhibitors:

1. The limitation of IPv4 in terms of the number of physical addresses that were available
2. The capacity of the internet for a fully connected IoT
3. The ability for mediators to scale to millions of concurrent connections
4. The ability to  store and analyse the data in a scalable way
5. The ability to analyse all the data to make sensible decisions in a timely manner.

Fast forward to today and we have most of these things from a technology perspective either solved (e.g. IPV6) or the pieces are available, and Red Hat is ideally placed to provide the whole solution for a scalable backend for the IoT - and to do it all on open source software.

Firstly, we need the ability to provide a standards based, horizontally scalable solution for handling connectivity to hundreds of thousands of concurrent connections. JBoss A-MQ is combining the best of Apache licensed middleware solutions from Apache ActiveMQ, QPid and HornetQ to form a highly scalable messaging solution that supports MQTT, AMQP,  WebSockets and STOMP.

The IoT will generate a lot of unstructured data, which needs to be correlated and analysed, and one of the leading NoSQL solutions for doing this is Hadoop. If you want Hadoop to scale and perform, then bester infrastructure to run it on is a combination of GlusterFS and OpenStack.

Getting real time data into Hadoop's HDFS can be problematic, but  JBoss Fuse already has some of the best solutions for doing just that.

Finally, if you want to use complex event processing, to make decisions based on the flow of data from your connected devices based on causality and temporal logic, then JBoss BRMS is the best open source solution on the market.

Red Hat  is going to be right at the centre of  IoT solutions in 2014.