DB Deployments: Using A Custom Filter

So I’ve been rolling out the deployment pipelines as I was able and I ran into an issue with our third one: We have a helper database that lives on all of SQL Instances and while we’d like to eventually keep any customization in source control we only want the dbo schema deployed automatically via the pipeline. The Redgate “Deploy From Package” step has a handy option to include an alternative filter to use with the SQL Compare engine so it seemed really straight forward: I fired up SQL Compare: generated a unique filter file that I named DeployFilter.scpf and threw that file name into the job step!

deployfilter

Unfortunately I got stuck here because this does not work. The nuget packages don’t end up in the working directory for the deployment and I couldn’t find any documentation about where it might end up. I ran a few deployments so I could watch the working folder as the build ran to see what showed up with no luck. Looked for articles etc. that might have mentioned configuring the path (to no avail). Eventually I hit up the Redgate channel on the SQL Slack. Alex Yates (b|t) had the answer in his head all along: The previous step (self-deploy package) unpacks the nuget package on the tentacle and I have to get it to tell me where the package is (and then append the path to the filter within the package itself). This is all very long way of saying if you want to use a custom filter (a) it should be in source control with the rest of your code and (b) the variable is #{Octopus.Action[<name of package deployment step>].Output.Package.InstallationDirectoryPath}\<path to filter file in nuget package>. And I am putting it here for the next person that finds themselves googling how to make it work (which may still be me: my buffer is notoriously overworked).

DB Deployments Part 2: Configuring Octopus Deploy

Where Team City will test and validate your code, Octopus Deploy will manage where that code is deployed. Again there are SQL Change Automation tools we’ll be using to ensure our database code is deployed consistently across our environments. You’ll need to ensure you have a service account (or MSA/gMSA eventually) for the application as well as home for the SQL database it needs. Download the install media and deploy it to your server. We’ve have Octopus deployed side by side with Team City (just on different ports).

Once installed under configuration you can add users and groups to roles as needed. We’ve created groups for the DBA and BI teams so they can log in and see the status of their deployments and added folks to the Administrators group as appropriate. (A quick note: while there is a small teams license for Octopus all users in the app are essentially an admin unless you purchase licensing: You’ll very likely want to purchase a license to take advantage of role based permissions.)

Screen shot of the Octopus Deploy Infrastructure page. It shows 4 environments and 1 deployment target.

A quick peek at our Octopus Deploy Infrastructure

The next piece that needs configuring is Infrastructure. We need to define agents that will do deployments (called deployment targets or tentacles) and define deployment environments. Again we don’t expect our deploy agents to be very busy to start with so we have a single agent that also runs on the app server (just like Team City). Clicking Add Deployment Target Octopus will walk you through installing the agent on a new server: it was pretty painless. At this time we’ll also create a named role for the tentacle that indicates what it might be used for (this is useful if your deployment process spans different servers and you need more than one agent to deploy your code). We only ever deploy to SQL servers so again a single agent and role is sufficient for now. Once we have a target we can define Environments and assign a target that will handle work in that environment. If your environments are highly segmented you may need a target (not on the app server) that is on the right network with the right connection rights (to whichever environment) that the app server can communicate with to do deployments. I have 4 environments: DBA Dev, DBA Test, DBA Stage and DBA Prod that match up with the deployments I will want Octopus to do.

Screenshot of the Octopus Deploy Variable Sets menu

Octopus Deploy Variable Set: DBA Deployment Locations

Next we add some objects to the Library tab. Under Variable Sets I am going to define a set of variables called DBA Deployment Locations. These are a standardize variable that we can include in Octopus Projects. I’ve defined the database name and SQL Instance as well as connection string based on those two values. Secondly for Step Templates I browse the library to down and install the Redgate step templates for creating, deploying releases. Additionally I grab the SQL – Execute Script template. I’ve also added two lifecycles that control deployment behavior: DBA Dev which deploys automatically to both the DBA Dev and DBA Test environments and DBA Production with automatically deploys to DBA Stage and is linked to DBA Prod. Lastly you’ll note there is a packages section: this is where Octopus will display packages that have been passed to it. Now we are ready to create our first project!

Projects can be put into groups and I’ve continued the grouping of DBA vs BI projects here as well. Once all that is in place we can create a project to handle specific deployments. Under the project tabs there is an Add Project button. The project needs to have the same name as the Team City project for ease of interoperability. I give it a description and assign it to a group and a Lifecycle. Once on the project page I expand Variables from the navigation pane on the left and click Library Sets. Then in the upper left I click “Include Library Sets.” I choose “DBA Deployment Locations” I just created and click save. I need to create some project variables as well so I can assign specific values to that variable based on the environment the tentacle is connecting to. The really nice thing is that I can overload existing variables I made have added from library sets so I recreate my DB Name and SQL Instance variables. I have different database names and SQL Servers for different environments and I input them all here. The scope for variables is very useful and available just about everywhere in the UI. The nice feature is that you can preview variable values at specific process steps to ensure they are being set properly throughout the deployment process.

I then create two channels to handle the releases and assign a lifecycle to each: Develop gets the default DBA Dev Lifecycle and then Master is associated with the DBA Production Lifecycle. The last bit is to create an actual deployment process. After clicking Process on the left navigation pane under the project we can start adding steps. I have six (although they don’t all run in every environment).OctoProcess

    1. The first step will create the database if it does not exist on the SQL instance already. I select “SQL – Execute Script” from the templates available and configure it. I only have one choice of target roles (my single tentacle that is configured) and I select that and then provide the necessary components.
      • Connection string generated based on the SQL Instance variable
        server=#{SQL Instance};database=master;integrated security=true
      • The SQL script itself which can again be customized with the Octopus variables:
        IF NOT EXISTS ( SELECT                *
        FROM   databases
        WHERE name = ‘#{DB Name}’)
        BEGIN
        CREATE DATABASE [#{DB Name}];
        END;
      • I check the box to marking this step as required for every deployment. If the database isn’t there the rest of the deployment will surely fail.
    2. If the database does exist (particularly in staging or prod) I want to back it up before I potentially change it. I add another execute SQL step that backs up the database. When configuring the step I add expand the Conditions section and select “Run for only specific environments” and then only include DBA Stage and DBA Prod. I also require this step before any further steps can be run. The SQL script runs our local backup procedure for the database contained in the variable #{DB Name}.
    3. The next step passes the deployment package to the tentacle. Under Package Details we leave the Package Feed as Octopus Server (built in) and give it the name of the project for the Package ID. (I also un-check everything under the Features section as I know we are not using them.) I also ensure the step is required.
    4. This step is the actual deployment from the nuget package. This step template is Redgate – Deploy From Package. I need to give it a few things to work.
      • It needs to know which step grabs the package it needs so we point it at step 3.
      • It needs the SQL Instance variable for the target SQL instance as well as the target database name. Optionally we could configure a user name and password but our build agent service account will have the appropriate permissions on the SQL instance.
      • I do give it SQL Compare options (IgnoretSQLt, DecryptEncryptedObjects1)
      • Again this step cannot be skipped.
    5. If the deployment fails outside of production we’d like to get an email. I configure an email step that is limited to all environments except DBA Prod: super easy since the Environments conditions has a “skip environments” option. Also I limit the step to run only if a previous step has failed.
    6. If the deployment fails in production we want someone to be paged: I have a second email step that sends an email to Pagerduty address set up to alert the DBA on call and is restrictued to only the DBA Prod environment. Otherwise it is the same as step 5.

 

Now that the project is configured I can go back to Team City and create a build project and start triggering builds. But that is a post for another day.

1. If you have encrypted objects in your database you’ll need this second option as well as to grant the deploying user SA on the SQL Instance in question otherwise your deployment will fail as SQL Compare will not be able to validate the deployment because it won’t be able to compare encrypted objects, this goes for Team City as well.

Database Deployments for Non-Developers

Ever since my first disaster of a manual deployment as a DBA I’ve been on the DevOps train. Maybe I didn’t know exactly what it was called but certainly this is a solved problem I said to myself? How do other organizations find the changes that need to be deployed from their dev/test environment into production? How can I ensure that we have a process that has the best chance of succeeding when we go to deploy? And most importantly: can I get out of logging in after hours to manually run scripts?

This, dear reader, is what brought me to Redgate originally: the SQL Compare tool was exactly what I was looking for in early 2014. Working backwards I started requiring that we wrap tooling around our SQL deployments. It started small (and slow): prior to deployments developers needed to meet with me to go over the change report to identify production ready changes they wanted deployed but ended big: last year the DBA team was out of the deployment game altogether since the development team had implemented SQL Change Automation steps in Team City and Octopus Deploy. But not so fast: what about the database development not done by the development team? I am sad to report that the DBA owns its own data repository, helper databases on every SQL instance and the occasional one off database that ends up under our care. Additionally our BI team fully owns the development of our data warehouse and staging databases. Few of these were in source control and in the case of our help db there was nothing in place to ensure they were up to date across our environment. For various reasons we decided we needed to own our deployment tools and we recently found ourselves with our own Team City and Octopus Deploy server. The next couple of blog posts will document the places where I had to do some quick learning/trouble shooting to get it all working right since: I have not been in the development world since 2003 and having solely been a production DBA since my return to tech in 2013. Hopefully if you are doing database development—but aren’t what folks would consider a traditional developer—this guide will prove helpful for standing up and taking advantage of what all this automation is able to offer you!

  1. Configuring Team City
  2. Configuring Octopus Deploy
  3. Building a deployment pipeline.

Data Privacy Is Coming For You

I’ve been thinking a lot about GDPR and similar legislation for the last year or so. I think that codifying how companies treat our data (and what they can and can’t do it with) is a huge step in the right direction. Late in 2018 I was happy to find myself working with Redgate on their SQL Data Catalog early access program as we started looking at ways to attach and track metadata about our data in our environment. Last month Washington state (where I currently reside and work) proposed it’s own GDPR like legislation. Since we aren’t likely to get any federal level legislation anytime soon it’s most likely the US DBAs will have to learn about a variety of laws and investigate (along with a legal team) what they actually mean and if they impact their organization.

In the mean time now we can still a jump start on collecting the data we need to meet what is sure the bare minimum requirement of these laws: getting a handle on what you actually have in a way that is accessible and reportable. Also reviewing your organizations practices for cleaning data in dev and test environments. This means generating test data or masking/altering production data to remove/replace sensitive information. If digging into market trends to see what other organizations are doing is your thing Redgate is making the 2018 Data Masking Market Guide available for your perusal here. I’m really enjoying my foray into these questions! Have you started trying to solve these problems in your org? What have you run into that you weren’t expecting when you started?