I met a strange problem recently. After configure the CI/CD pipeline with Azure DevOps and automatically deploy to Azure App Services, the. NET Core Website failed to start. Let's see how we can solve this problem.
Find the issue
First of all, fortunately, this is a staging environment. I found that the site unexpectedly did not log anything, even log directory can not be found, which is very strange. So I decided to enable development mode in Azure so that the site can output detailed error messages. If you are familiar with the classic ASP.NET, you may know that changing customError in web.config can show detailed error. But .NET Core doesn't have this thing, how can we do this? There's a small trick:
The settings of .NET Core can be overwritten with environment variables. So we can take advantage of this by adding an environment variable to the settings of Azure App Services:
It is located in Application settings page.
How could I know this? Actually it is from Visual Studio itself:
Now, we can see detailed error:
I can see that the code that get executed is not in the root directory of the site. So my code couldn't find the dependency, and it blew up.
What is going on?
I tried to deploy manually from VS, and it blew up, too. Redeploy in Azure DevOps, blew up. Even if I deactivate the CD pipeline, use kudu to build git code directly, also blew up!
Eventually I deleted the site and created a new App Service instance, published website from VS, and it was good. But once again when I deployed it with the CI/CD pipeline, it blew up again! I compared the settings from the previous good website, and found there's a new settings showing up:
What the heck is this?
I traced it back to a Microsoft Azure announcement: https://github.com/azure/app-service-announcements/issues/84
Today, App Service supports a number of different techniques to deploy your files: msdeploy (aka WebDeploy), git, FTP, VSTS, ARM and more. But all these techniques have something in common: when all is said and done, your files are deployed under your wwwroot folder (specifically `d:\home\site\wwwroot`).
At that point, the runtime takes over, and it is no longer relevant how the files got there. The runtime can read all those files, and run whatever logic it needs to run based on them.
But with Run From Package, things are very different. There is no longer a deployment step which copies the individual files to wwwroot. Instead, you just point it to a zip file, and the zip gets mounted on wwwroot as a read-only file system.
So this is why my website runtime root directory is changed.
Recover the website
To temporarily recover the website to run, very simple, just delete WEBSITE_RUN_FROM_PACKAGE from setting, restart the site, you can revert to a good pre-deployment version. However, future deployments of Azure DevOps will not work. Because the real wwwroot directory will not be updated.
(website recovered to pre-deployment version 10.0.6980.1000)
But, we still need automated deployments
We need to modify the default values for Azure DevOps deployment tasks.
Edit your Release definition, go to Tasks > Deploy Azure App Service
Expand Additional Deployment Options and select Select deployment method and use Web Deploy
Now, issue a new release, wait for it to finish, you will have a good new version deployed at wwwroot directory correctly!