In a previous post we explained how to Install and Configure Apache Airflow (a platform to programmatically author, schedule and monitor workflows). The technology is actively being worked on and more and more features and bug fixes are being added to the project in the form of new releases. At some point, you will want to upgrade to take advantage of these new feature. In this post we’ll go over the process that you should for upgrading apache airflow versions. Note: You will need to separately make sure that your dags will be able to work on the new version of Airflow. Upgrade Airflow Note: These steps can also work to downgrade versions of Airflow Note: Execute all of this on all the instances in your Airflow Cluster (if you have more then one machine) Gather information about your current environment and your target setup: Get the Airflow Home directory. Placeholder for…
Month: March 2017
Continuous Delivery With GoCD
This blog outlines our experience moving one of our projects to Continuous delivery Model using GoCD on AWS. Prior to this implementation our code deployments were manual and on demand. We were looking for automated way of deploying code to various environments with minimal manual intervention. GoCD has continuous delivery as a first-class concept and provides an intuitive interface to start building CD pipelines. We started off with a quick PoC to validate some of our understanding and after initial success, we now use GoCD to define all of our deployment/delivery pipelines. This move forced us to have comprehensive test suite and work flow defined to set the criteria to promote code in different environments. The change also increased our ability to push more but smaller changes frequently. Deployment vs Delivery It is fairly often to see the terms Continuous Delivery and Continuous Deployment used interchangeably. For some this is a huge distinction…