ASP.NET Core 2.1 Re-target to Full Framework 4.7.1

I am in the middle of upgrading one of my websites from ASP.NET 4 to ASP.NET Core 2.1 using Razor Pages and I absolutely love everything about the new development experience. I also upgraded all my libraries to use an appropriate .net standard version, but I was unable to upgrade one project due to the differences in how WindowsAzure.Storage nuget package is used when targeting full framework vs .netstandard. Too much had to be re-written so I just left that one library targeting full framework. However, when I tried to bring that dependency into the ASP.NET Core 2.1 site, it would not work since I was targeting netcoreapp2.1 and the WindowsAzure.Storage package ran into "Missing Method Exceptions". At this point, I had to either update the one library or update the site to target full framework. Long term, I want to update the library. Short term, I need to get past this problem, so re-targeting it is.

Retargeting

I had hoped that I could just change the TargetFramework element in the project file to net471, but that did not work. I was getting an error Package Microsoft.AspNetCore.App 2.1.0 is not compatible with net471 (.NETFramework,Version=v4.7.1). Package Microsoft.AspNetCore.App 2.1.0 supports: netcoreapp2.1 (.NETCoreApp,Version=v2.1) From there, I figured there were a different set of packages, so I created a new project targeting full framework and compared the project files. I noticed the following packages that I needed to use instead of Microsoft.AspNetCore.App. So here's what I had to change (note, I'm using a different Identity provider so I removed Entity Framework packages that were included by default):

Target Framework: netcoreapp2.1 => net471
Packages: Microsoft.AspNetCore.App =>
"Microsoft.AspNetCore" Version="2.1.1"
"Microsoft.AspNetCore.Authentication.Cookies" Version="2.1.1"
"Microsoft.AspNetCore.CookiePolicy" Version="2.1.1"
"Microsoft.AspNetCore.HttpsPolicy" Version="2.1.1"
"Microsoft.AspNetCore.Identity.UI" Version="2.1.1"
"Microsoft.AspNetCore.Mvc" Version="2.1.1"
"Microsoft.AspNetCore.StaticFiles" Version="2.1.1"

After these updates to the project file, I was able to compile and run and my full framework library was able to be used with no problem.

Knockout datalist binding when input selected

I recently had the need to know when an item from a datalist was selected vs a user entering text into the input so that I could trigger a different behavior in the UI. This is what I came up with based on the 'input' event that is raised. When raised due to the user typing, the event's originalEvent property is an InputEvent and has an inputType property. If the user selects a value, the event's originalEvent property is an Event and does not have that property. Not sure if there's a better way, but this works! :)

    /**
     * @desc This binding will trigger when a user selects an item from
     * the data list and will pass the selected value to the specified function
     *
     */
    ko.bindingHandlers.datalistInput = {
        init: function (element, valueAccessor) {
            $(element).on('input', function (e) {
                if (e && e.originalEvent && e.originalEvent.type === "input" && !e.originalEvent.inputType) {
                    var functionToExecute = ko.utils.unwrapObservable(valueAccessor());
                    if (functionToExecute && typeof (functionToExecute) === 'function') {
                        functionToExecute(e.target.value);
                    }
                }
            });
        }
    }

This can be used with an input and datalist like this:

    <input class="form-control"
           data-bind="textInput: userInput,
                      datalistInput: function(selectedText) { console.log(selectedText); }"
           list="mydatalist">
    <datalist id="mydatalist">
        <!-- ko foreach: listOptions -->
        <option data-bind="text: $data"></option>
        <!-- /ko -->
    </datalist>

In this case, textInput will bind to anything the user enters as well as any value selected from the datalist by the user. The datalistInput will only be triggered when the user selects a value from the datalist.

Copy all Azure Tables from one storage account to another

As a follow up to my previous post about copying blobs, here's how you can copy tables. This is even more rough around the edges than copying blobs. Firstly, you cannot just copy the table; you have to export it, then import it. Further, this cannot be done all in azure. When exporting, even if you use the command to export to blob container, it will download locally, then upload to the blob container so this can be quite a bit slower and incur additional charges so make sure you know what you're getting into.

The Script

cd 'C:\Program Files (x86)\Microsoft SDKs\Azure\AzCopy'

$sourceStorageAccountName = "SOURCE_STORAGE_ACCOUNT_NAME"
$sourceStorageAccountKey = "SOURCE_STORAGE_ACCOUNT_ACCESS_KEY"

$destStorageAccountName = "DESTINATION_STORAGE_ACCOUNT_NAME"
$destStorageAccountKey = "DESTINATION_STORAGE_ACCOUNT_ACCESS_KEY"
$destTemporaryContainerName = $(((Get-Date -Format o) -Replace '[^a-zA-Z0-9]','').ToLower())

$sourceStorageAccount = New-AzureStorageContext -StorageAccountName $sourceStorageAccountName -StorageAccountKey $sourceStorageAccountKey
$destStorageAccount = New-AzureStorageContext -StorageAccountName $destStorageAccountName -StorageAccountKey $destStorageAccountKey

$tables = Get-AzureStorageTable -Context $sourceStorageAccount
foreach($table in $tables) {
 Write-Host "Copying source table $($table.Name) from $($sourceStorageAccountName) to temporary storage container $($destTemporaryContainerName) on $($destStorageAccountName)"
 .\AzCopy.exe /Source:https://$sourceStorageAccountName.table.core.windows.net/$($table.Name)/ /Dest:https://$destStorageAccountName.blob.core.windows.net/$destTemporaryContainerName/ /SourceKey:$sourceStorageAccountKey /Destkey:$destStorageAccountKey /Manifest:"$($table.Name).manifest"
 Write-Host "Finished copying source table $($table.Name) from $($sourceStorageAccountName) to temporary storage container $($destTemporaryContainerName) on $($destStorageAccountName)"
 
 Write-Host "Importing data into destination table $($table.Name) from temporary storage container $($destTemporaryContainerName) on $($destStorageAccountName)"
 .\AzCopy.exe /Source:https://$destStorageAccountName.blob.core.windows.net/$destTemporaryContainerName/ /Dest:https://$destStorageAccountName.table.core.windows.net/$($table.Name)/ /SourceKey:$destStorageAccountKey /DestKey:$destStorageAccountKey /Manifest:"$($table.Name).manifest" /EntityOperation:"InsertOrReplace"
 Write-Host "Finished importing data into destination table $($table.Name) from temporary storage container $($destTemporaryContainerName) on $($destStorageAccountName)"
}

Write-Host "Deleting temporary storage container $($destTemporaryContainerName) on $($destStorageAccountName)"
Remove-AzureStorageContainer -Context $destStorageAccount -Name $destTemporaryContainerName -Force
Write-Host "Finished deleting temporary storage container $($destTemporaryContainerName) on $($destStorageAccountName)"

Copy all Azure Blob containers from one storage account to another

If you're using Azure Storage, you'll probably want to be able to clone storage account for testing purposes, setting up multiple environments, etc. To my surprise, there is not a very straight forward way to just clone a storage account.

AzCopy

AzCopy is a utility that helps move data to and from storage account. It has a method to copy data between storage accounts, but can only do a single container at a time. Not sure why they couldn't just make it do all containers, but it doesn't, so what are our options. Well, if you're developing for Azure, then PowerShell is probably going to be your best bet. You could technically write the entire transfer script using PowerShell, but AzCopy does a good job copying per container so let's just use PowerShell to iterate all the containers and have AzCopy do the work.

The Script

cd 'C:\Program Files (x86)\Microsoft SDKs\Azure\AzCopy'

$sourceStorageAccountName = "SOURCE_STORAGE_ACCOUNT_NAME"
$sourceStorageAccountKey = "SOURCE_STORAGE_ACCOUNT_ACCESS_KEY"

$destStorageAccountName = "DESTINATION_STORAGE_ACCOUNT_NAME"
$destStorageAccountKey = "DESTINATION_STORAGE_ACCOUNT_ACCESS_KEY"

$sourceStorageAccount = New-AzureStorageContext -StorageAccountName $sourceStorageAccountName -StorageAccountKey $sourceStorageAccountKey
$destStorageAccount = New-AzureStorageContext -StorageAccountName $destStorageAccountName -StorageAccountKey $destStorageAccountKey

$containers = Get-AzureStorageContainer -Context $sourceStorageAccount
foreach($container in $containers) {
 Write-Host "Copying container $($conatiner.Name) from $($sourceStorageAccountName) to $($destStorageAccountName)"
 .\AzCopy.exe /Source:https://$sourceStorageAccountName.blob.core.windows.net/$($container.Name) /SourceKey:$sourceStorageAccountKey /Dest:https://$destStorageAccountName.blob.core.windows.net/$($container.Name) /DestKey:$destStorageAccountKey /S
}

Durability

If for some reason your script terminates, just run the same command that just terminated and you'll be prompted whether you'd like to resume or restart the transfer. Cool!

Restrict access to your Azure App Service to users in your Office 365 Active Directory

Recently I had the need to restrict access to my Asp.net core 2.0 application to only users in my Office 365 subscription. I was not able to find any good documentation of how to do this; everything I could find was outdated and didn't match with the screens I encountered. After struggling thru it, I finally got it working and here's how.

Office 365 & Azure Active Directory

The first thing that may not be obvious at first is that Office 365 uses Azure Active Directory to manage users. Because of this, any documentation referring to authentication with Azure Active Directory (AAD) pertains to Office 365 authentication.

• AD Auth using old portal
• App Service Authentication (EasyAuth) using old portal

These were the closest references I could find that eventually got me going, but they are against the old portal and using Azure not Office 365. I couldn't find anything that showed how to do it via Office 365.

Register the Application

As with all the tutorials, let us start by registering the application in Office 365. To do this, we'll need to get into AAD associated with the Office 365 subscription.
One way to do this...

1. Login to Office 365
   
2. Go to admin portal
   
3. From the navigation, expand Admin Centers and select Azure AD

App Registration

From the Azure Active Directory dashboard, pick App Registrations if it is visible in the left navigation, otherwise expand more services and select it there. You can star it to add to the left navigation if you want.

New application registration

From the App Registrations blade, select New application registration. Provide a meaningful name for the application. You can enter the site's production url (note: make sure to prefix https:// if your site is secure) here with the suffix .auth/login/aad/callback to help fill in the next blades. Tab out of the field to enable the Create button. Click it and select the newly created application.

Reply Urls

The Sign-on URL provided during creation is automatically added to the Reply URLs for this application. One of the nice things about the registration is that we can specify multiple reply urls that are valid for this application. This allows us to specify staging, uat, testing, development urls that can all share this authentication (of course, be smart about it). Go ahead and add any you want now. You can always add more later.

Api Access Key (Client Secret)

Now select the Keys tab to create a key that our web app can use to identify itself to AAD. Enter a name for the token and select a duration for which the token will be valid. After saving, the secret will be shown. Make sure to grab it right away and save it somewhere safe. It will not be visible once you close that blade. This will be used as the client secret when enabling authentication.

App Service Authentication

Because I simply needed to know if the user is in my active directory and nothing else, App Service Authentication is the simplest way to get this going (of course, after hours of hair pulling due to lack of documentation; but now it's a breeze for future sites :D ).

Enable Authentication

Pick the Authentication / Authorization menu item and turn on App Service Authentication. Then select "Login with Azure Active Directory" from the "Action to take when request is not authenticated" dropdown.

Advanced Configuration

From the listed Authentication Providers, select Azure Active Directory. For the management mode, select Advanced. For the client ID, enter in the Application ID of the newly registered application.

Issuer Url

This was by far the most annoyingly difficult piece to figure out. It will be https://login.microsoftonline.com/{tenant-id} where {tenant-id} is the directory id. To get this url, go back to Office 365 App Registrations, and select the Endpoints top menu item. Copy any of the endpoints and delete everything past your tenant-id, which will be a GUID. (Ex, https://login.microsoftonline.com/xxxxxxxx-eeee-4b8a-a886-xxxd4xxxxxxx/federationmetadata/2007-06/federationmetadata.xml would be https://login.microsoftonline.com/xxxxxxxx-eeee-4b8a-a886-xxxd4xxxxxxx).

All said and done, it should look something like this:

Having all these steps outlined definitely makes it super simple to restrict access to an azure app service to users in an Office 365 subscription using all new portals!

Create Azure Function Queues Automatically

I have been spending a lot of time working with Azure Functions lately and I really enjoy it, but there are a few pain points still. Automatically creating resources used by the Azure Functions does not seem to be in-the-box. Often, the functions you write need queues or other resources to function properly; meaning the queues may be an implementation detail to the azure functions. If you are a believer in continuous integration and deployment, then you probably have a way to deploy your functions anywhere, but...

Azure Resource Manager

Using Azure Resource Manager (ARM) templates, you can automatically deploy a service plan (consumption or reserved), and deploy your azure functions to that plan. You can also create a storage account and put the connection string to that account into the app setting of the deployed azure functions. However, you cannot create the queues or containers needed by the functions via ARM templates.

PowerShell

One can always take it upon themselves to write a powershell script that grabs the connection string and creates the resources, but now you have to maintain a powershell script with the same names used in your azure functions. This results in two places where magic strings needs to be in sync and while it is technically code, it is an unnecessary break from the programming model we are already using.

Reflection

What I tend to do instead, is some follow some basic conventions and use reflection over my azure functions. Here is what the top of most of my functions look like:

// Storage Helper in another file...
internal static class StorageSettings
{
 public static string ConnectionKey = "myStorageKeyInAppSettings";

 public static CloudStorageAccount StorageAccount { get; } = CloudStorageAccount.Parse(ConfigurationManager.ConnectionStrings[ConnectionKey].ConnectionString);

 public static Lazy<CloudBlobClient> BlobClientLazy = new Lazy<CloudBlobClient>(StorageAccount.CreateCloudBlobClient);
 public static CloudBlobClient BlobClient => BlobClientLazy.Value;

 publicstatic Lazy<CloudQueueClient> QueueClientLazy = new Lazy<CloudQueueClient>(StorageAccount.CreateCloudQueueClient);
 public static CloudQueueClient QueueClient => QueueClientLazy.Value;
}

// Functions class that has Queue resource dependencies
public static class Functions
{
 static Functions()
 {
  System.Diagnostics.Trace.TraceInformation($"Creating queues used by {nameof(Functions)}.");
  Task.WhenAll(QueueNames.Value.Select(x =>
   {
    System.Diagnostics.Trace.TraceInformation($"Creating queue {x} if not exists.");
    return StorageSettings.QueueClient.GetQueueReference(x).CreateIfNotExistsAsync();
   }))
   .GetAwaiter()
   .GetResult();
 }

 private const string StepOneQueueName = "step-one";
 private const string StepTwoQueueName = "step-two";

 private static readonly Lazy<string[]> QueueNames = new Lazy<string[]>(() =>
  typeof(Functions).GetRuntimeFields()
   .Where(x => x.IsLiteral
      && x.FieldType == typeof(string)
      && x.Name.EndsWith("QueueName"))
   .Select(x => (string)x.GetValue(null))
   .ToArray());

 public static async Task StepOneAsync(
  [QueueTrigger(StepOneQueueName, Connection = StorageSettings.ConnectionKey)] string stepOneMessage,
  [Queue(StepTwoQueueName, Connection = StorageSettings.ConnectionKey)] IAsyncCollector<string> stepTwoMessageCollector,
  TraceWriter log)
  {
   /* do work here */
  }
}

As you can see, there is a static helper class that has common information about shared storage accounts (in this case, I just have one, but you could have configuration classes for each storage account). By making the *QueueName and ConnectionKey properties constants, you can use them as attribute values. This removes any chance of fat fingering the queue names or connection keys. In the static constructor of the Functions class, I'm simply looking for any constant whose name ends with QueueName and is a string type. Then I just create all the queues if they do not exist. This keeps the creation of the queues in the same location as the function and is automated.

Xamarin.Android build Task + $(SolutionDir)

I was recently setting up build automation for a Xamarin.Forms project I'm working on and found that the initial template was not able to build my project. I was getting an error like:

C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\MSBuild\15.0\Bin\Microsoft.Common.CurrentVersion.targets(1987,5):
warning MSB3245: Could not resolve this reference. Could not locate the assembly
"Newtonsoft.Json, Version=9.0.0.0, Culture=neutral, PublicKeyToken=30ad4fe6b2a6aeed, processorArchitecture=MSIL".
Check to make sure the assembly exists on disk. If this reference is required by your code, you may get compilation errors.
For SearchPath "{HintPathFromItem}".
Considered "*Undefined*packages\Newtonsoft.Json.9.0.1\lib\portable-net45+wp80+win8+wpa81\Newtonsoft.Json.dll", but it didn't exist.

Notice the *Undefined* in the considered path.

After a little digging, here's what I found.

In all the other projects in the solution, I include packages using the $(SolutionDir) variable in the HintPath like:

<Reference Include="Ninject, Version=3.0.0.0, Culture=neutral, PublicKeyToken=c7192dc5380945e7, processorArchitecture=MSIL">
  <HintPath>$(SolutionDir)packages\Portable.Ninject.3.3.1\lib\portable-net4+sl5+wp8+win8+wpa81+monotouch+monoandroid+Xamarin.iOS\Ninject.dll</HintPat>
  <Private>False</Private>
</Reference>

Because the Xamarin.Android task is building a project, there is no $(SolutionDir) available. If you use the VisualStudio build to build the Xamarin.Forms solution, it will build find. So how to get Xamain.Android to replace the $(SolutionDir) variable?

Add a Build Variable!

To figure out the value of the build variable, at the top of the logs for the Xamarin.Android build step you'll find a few lines like this:

Build started 9/18/2017 8:59:48 PM.
Project "d:\a\3\s\DEV\App.Mobile.Droid\App.Mobile.Droid.csproj" on node 1 (PackageForAndroid target(s)).

In my case, the directory was d:\a\3\s\DEV\ because in the Get Sources step, I specified DEV as the folder to put the sources into. If you don't do this, it will typically be something like c:\a\1\s\
At any rate, after adding a build variable with this value, the Xamarin.Android task is able to properly find the packages required to build the solution.