Part 2 – https://verticalagetechnologies.com/index.php/2023/06/28/azure-packer-powershell-creating-citrix-golden-images-part-2 (json/HCL)
Part 3 – This post
Hopefully by now you have a decent idea on how to structure your azure packer deployment. For me this was pretty challenging to start. Seemed I had to visit 20 different sites to find examples of code that isn’t documented well (hint hint Hashicorp), as well as other errors along the way. Below I’ll talk about some of the gotchas I encountered throughout the process.
Time
The task sequence process I showed you in Part 2 is almost a mirror of what I’m also doing in MDT on prem, which takes around 2.5 hours from total start to finish. This includes building a VM in Nutanix, installing OS from scratch, laying down all the apps, sealing, creating PVS image, booting VDA and confirming it gets registered, and lastly sending a completion notification. I noticed that some of the application installs I did on prem took 10 minutes, but in Azure they were taking 1 hour. Adobe Acrobat being one of them, .Net being another. This is only with 30ms latency via Express Route as well. So as a general best practice, try to install your applications locally if possible. Meaning either download/install them, or sync the installs up to azure, then install from there.
Storing of Secrets/Passwords
Just feel like there’s no one way or best way to accomplish this. if you are just running your scripts manually (not automated), then prompting for the creds is the best way. If you are trying to automate/schedule the process, then you don’t have much of choice, they need to be stored somewhere. Just make sure they are stored securely. I chose a few different methods, some being better than others. I’d choose the Vault method over the encrypted NTFS file method. Be cognitive of service accounts being used and if you store them in a vault, that they are included in the password rotation process.
Elevation in PowerShell Provisioner
If you take a look at the PowerShell Provisioner section in the HCL file, you’ll see most of the sections have a lines that say:
elevated_user = "Administrator"
elevated_password = "${var.AZURE_LOCALUSER}"
This is so Packer can run the PowerShell script as an elevated user. When normally using the PowerShell provisioner without this elevation code, Packer will run the script as a Windows Scheduled Task under normal ‘WinRM’ account you chose. Sometimes there will be code that you are running, that requires elevation (right click – run as admin type actions). For me, this was a decent amount of things, especially when installing Applications from a network share. For instance, I couldn’t map a Nutanix Files share without this elevation code. However, I could map a normal Windows Share. When in doubt, if something isn’t working or getting installed the way you want it to, try using elevation.
From Scratch Builds
Most of my Golden Image experience thus far has been with MDT (Microsoft Deployment Toolkit). While it hasn’t had any extreme makeovers or many updates over the past decade, it has always been a crutch in a tried and true scenario. I can’t tell you how much my world changed in the evolution supporting images. From standalone builds, to standard non persistent image done with PVS, to AppLayering, and now to building from scratch images, whether it’s via MDT, Packer, Terraform, etc… My admin time supporting the images and applications inside it dropped significantly, not to mention the advantages when moving across hypervisors, or platforms/clouds, or Operating Systems, etc…
Exit Codes
Be mindful of the $exitcodes that your PowerShell scripts generate. Most will have an LastExitCode of 0, however, you’ll find a couple that just don’t fit the bill. You can always add the variable $LastExitCode to the end of the PowerShell script to confirm it’s last code. You may have to add this to the Packer PowerShell provisioner block via “valid_exit_codes = [0, 1, 3010, <insert random number>]”
Packer vs Terraform vs MDT
Pick your flavor. There’s not a right way to build and automate. It’s an easy out to say there are pros and cons for all, and that each use case is different. It Truly is. If you are on prem, MDT/SCCM is easy and there’s a ton of information out there. If you move to the cloud, don’t expect to be able to easily convert that task sequence though. Infrastructure as code is great for light weight deployment and infrastructure needs. It’s pretty much the only recommended way to build in the modern cloud. Azure itself uses Packer behind it’s “Azure Image Builder” feature. If you want to build a VM that you can use via #Citrix MCS, maybe Terraform would be the better option. Just all depends on the use case…..boring, I know.
Troubleshooting
Troubleshooting is so much easier when you can shoulder surf the console in an MDT/SCCM deployment. When using these more modern toolsets like Packer and Terraform it becomes a little harder. I continually found myself just redeploying everything. This is SLOW. If the Packer deployment fails somewhere, whether that be a bad exitcode or variable that wasn’t set right, or maybe it was set, but already exists, the Packer deployment goes to the last stage, which is cleanup. This goes through and cleans up everything it deployed (vaults, groups, VMs, etc…). So if won’t be able to login to the VM to see what happened. Then you have to kick off everything again from scratch and wait to see if it finishes successfully. If it’s PowerShell related and you know the spot, you can put a “start-sleep” in there to give yourself enough time to troubleshoot. Patrick Van den Born (@pvdnborn) clued me in on a way to just kick off the Provisioners and Skip the actual building. This is done via Null Builders: https://www.vandenborn.it/2022-08-24-null-builder/. This is also useful if Packer doesn’t support the environment you are trying to build on.
Closing
Hopefully a few of you found this information helpful and it wasn’t too redundant. If your stuck or have a question, or just want to get some opinions, head over to WorldOfEUC or MyCUGC slack channels. I promise you’ll get pointed in the right direction.
Part 2 – https://verticalagetechnologies.com/index.php/2023/06/28/azure-packer-powershell-creating-citrix-golden-images-part-2 (json/HCL)
Part 3 – This post