Slow Azure websites with Redis

During a load test of an Azure website, we noticed that the site started to become unresponsive when 10 concurrent users we loaded.

Session management has been utilized through a Redis cache. For anyone looking to use this I would recommend here as a good starting point.

Deploying the site locally using the live web.config transforms, we were able to run the following in the command line, to show all connections that were being made from the computer.

netstat -b

It was noticed that there were over 100 concurrent connections being made to the cache which stopped the site from being able to connect to the azure portal.

redisconnections_connectionerror

The azure portal had displayed the following recommendations to fix the portal

redisconnections_warnings

 

This was due to the concurrent connections being maxed out a 256 connections.

redisconnections_256

What was the culprit? Creating a new connection every time the class was instantiated.

It is recommend in the by MSDN to create a static instance of the connection that is shared across the application. This can either be achieved by using a singleton pattern, using dependency injection or by using the below code.

private static Lazy _redisConnection = new Lazy(() => 
ConnectionMultiplexer.Connect(<>));

This then happened.

redisconnections_10

 

Restricting your Umbraco web apps in Azure

Restricting access to a clients website can be a common practice. This usually means that they will need to have access give to an IP address or a range of IP addresses.

These details can usually be obtained from the clients IT administrators.

Typically during CMS development, you will have two instances of your application – delivery (Front end, with no access to the CMS) and an authoring (Front end and CMS access) and both of these instances will be found in each development environment.

Its usually good practice to restrict all access to development / UAT / SIT environments to your clients IP address. This allows for developments to be approved by the client before being released.

IP white listing will block all connections to the website in IIS unless it is specified on the list. Using a web.config transform would be the way to ensure that the restrictions get applied to the correct environment.

Step 1 – Create the white list. 

In App_Config create a new config file called ipSecurity.config

<?xml version="1.0"?>
<ipSecurity allowUnlisted="false">
</ipSecurity>

Step 2 – Local IP 

 <add allowed="true" ipAddress="127.0.0.1" />

Step 3 – Internal IP addresses

This will be your office IP address.

Step 4 – External IP addresses

This will be all of your stake holders in the project external to your company

Step 5 – Add your outbound IP Addresses

These are range of IP addresses that could be used, as Azure currently doesn’t support static outbound IPs. If you need a static IP, you could configure a custom domain.

Step 6 – Error handling 

Once locked down by IP, everything that interacts with the website will also be denied access including Application Insights.You will need to included all the IP addresses for Telemetry, Live Metrics Stream and Availability tests.
Availability web test is essentially a ping test that is run every few minutes to ensure your application is alive.

The range of up to date IPs that need to be added can be found here

https://docs.microsoft.com/en-us/azure/application-insights/app-insights-ip-addresses

Step 7 – Adding the list to web.config

Add the following code to you transform to apply the list.

 <system.webServer>
     <security>
         <ipSecurity 
           xdt:Transform="InsertIfMissing" 
           configSource="App_Config\ipSecurity.config" />
      </security>
 </system.webServer>

You should only apply this to the sites you need to restrict, with the exception of the public facing delivery application, otherwise your users wound be able to access the site! 😉

 

Creating thumbnails from the front page of a PDF with ghostscript

Automation is such as wonderful thing.

  1. install Ghostscript to your computer. https://ghostscript.com/download/gsdnld.html
  2. Create a C# console app
  3. Copy the following code into your app and replace <<VERSION NUMER>> with your version number.
  4. Pass in the link to the PDF file, the folder name and the file name.
 public void PdfToJpg(string inputPDFFile, string outputImagesPath, string filename)
 {
     string ghostScriptPath = $"C:\Program Files (x86)\gs\gs<<VERSION NUMBER>>\bin\gswin32.exe";
     string ars = $"-dNOPAUSE -sDEVICE=jpeg -dFirstPage=1 -dLastPage=1 -r102.4 -o \"{outputImagesPath}\\{filename}.jpg\" -sPAPERSIZE=a4 \"{inputPDFFile}\"";
     Process proc = new Process();
     proc.StartInfo.FileName = ghostScriptPath;
     proc.StartInfo.Arguments = ars;
     proc.StartInfo.CreateNoWindow = true;
     proc.StartInfo.WindowStyle = ProcessWindowStyle.Hidden;
     proc.Start();
     proc.WaitForExit();
 }

Distributed umbraco content

Lets assume that you have two umbraco instances

  • A delivery site – publicly accessible site with no access to the CMS admin screens
  • An authoring site – Usually an IP restricted site which has access to the CMS admin panel

Both of these sites will share the same data base and media library.

You have cached content within the application, anything from a list in a HTTP cache to media content and now the data has been updated in the CMS. You can see the content on the authoring site, but you cant see the content on the delivery site until the cache has expired.

There are two publishing events that you can use when updating content to refresh the caches on both instances

  • PageCacheRefresher.CacheUpdated
  • MediaCacheRefresher.CacheUpdated

 

Create a new ApplicationEventHandler in the site and add both of these methods in and expire your caches as required.

When the content is updated in the CMS on your authoring site, it will update the database and call events. This will then call your custom code to clear and rebuild your cache which will enable the delivery site to be updated when content is entered in authoring.

The documentation still needs updating here https://github.com/umbraco/UmbracoDocs/issues/324 

Azure search – Scoring profiles

A scoring profile allows Azure search to target and favor a combination of columns when searching for content.

Searching without a scoring profile lets Azure search calculate a score based on what it thinks the best match is. This produces a score against the search result and tells your what the most relevant item is.

To create a scoring profile, first log into the Azure portal and find the Azure search block and select the index of choice.

The opening blade will show you all of the properties of the index, including all of the scoring profiles. This is at the bottom.

azure_search_scoring_profile_100217
Available scoring profiles

Click the “Add scoring profile” button at the top

azure_search_scoring_profile_4_100217

name the new profile and a new profile will open in the blade

azure_search_scoring_profile_2_100217

Click on the weights button and select the filed and weighting against that field. The weights are between 0-10

azure_search_scoring_profile_3_100217

 

Save the profile to close the blade. In the prevous blade, click “Set as default” which will set the scoring as a default item. This means any search performed against the index will be run through this to ensure the results you are expecting match the criteria.

Wrapping an image with a link in Sitecore mvc

When wrapping an image with a link, you will need to pass the context into the field. In your view, put the following code.

 
@using Sitecore.Mvc.Presentation
@using Sitecore.Mvc
@model Sitecore.Sharedsource.Mvc.Presentation.RenderingModel

@Html.Sitecore().BeginField("Link", Model.Rendering.Item, new { haschildren = true })
 @Html.Sitecore().Field("Image", Model.Rendering.Item)
 @Html.Sitecore().EndField()

Adding expire headers for static content

Static content from a site can put unnecessary load when rendering content. Assets such as images, scripts and styles can be cached using an expire header to stop them from being requested on each HTTP request.

It would be wise to add a version number or date of the change to the end of each static item to allow the cache to be updated when new assets are added. For example

.png?v=1 or .css?v=01012017 .js?v=1

The cache can then be called either through setting a Common HTTP Response Headeror through code

Response.Cache.SetExpires(DateTime.Now.AddYears(1));
Response.Cache.SetCacheability(HttpCacheability.Public);

or by add the following code to your web.config

 <system.webServer>
 <staticContent>
 <clientCache cacheControlCustom="public" cacheControlMaxAge="12:00:00" cacheControlMode="UseMaxAge"/>
 </staticContent>
 </system.webServer>

Umbraco with https on Azure and scheduled published tasks

When forcing a site to run on HTTPS, the following rule is set in the web.config to force a site to 301 redirect to HTTPS when a HTTP link is found

  <rule name="Redirect to https" stopProcessing="true">
 <match url="(.*)" />
 <conditions>
 <add input="{HTTPS}" pattern="off" ignoreCase="true" />
 </conditions>
 <action type="Redirect" url="https://{HTTP_HOST}{REQUEST_URI}" redirectType="Permanent" appendQueryString="true" />
 </rule>

Whilst initial test will force the site to redirect to HTTPS, not all requests made from umbraco will execute using just this. We also need to ensure that the following key is also set

<add key="umbracoUseSSL" value="true" />

You must also make sure that set the location of the Umbraco endpoint in the umbracoSettings.Config

  <web.routing
 trySkipIisCustomErrors="false"
 internalRedirectPreservesTemplate="false" disableAlternativeTemplates="false" disableFindContentByIdPath="false"
 umbracoApplicationUrl="https://mysite.localhost/umbraco">
 </web.routing>

This will ensure that all requests made from umbraco to /umbraco/RestServices/ScheduledPublish/Index will be sent via HTTPS in the first instance.

When this is not set, a HTTP request is made which will result in the error

A public action method ‘Index’ was not found on controller ‘Umbraco.Web.WebServices.ScheduledPublishController’.

The error indicates that a request has been made via HTTP and redirected to HTTPS which has been rejected.

Once the App Setting has been set, the error will resolve.

 

Searching between dates using OData

Azure Search uses OData expressions syntax to build filter queries when searching within a searchable section of data.

Searching within a data range indicates that you should use a DateTimeOffSet, which is the data type stored in the, but it is easier to search using the following format

DateTime.Now.ToString(“yyyy-MM-dd’T’HH:mm:ss-00:00”);

To search between two dates, the filter list needs to be in the following format

$filter=year ge 2016-01-01T00:00:00-00:00 and year le 2016-12-31T23:59:59-00:00

If you are passing this via the REST API, URL encode the string before passing it to the service.
If you are using the .NET Azure package from NUGET, create a new Search Parameters object and pass the string to the Filter parameter and the encoding will happen automatically.

https://www.nuget.org/packages/Microsoft.Azure.Search 
https://docs.microsoft.com/en-us/rest/api/searchservice/supported-data-types
https://docs.microsoft.com/en-us/rest/api/searchservice/OData-Expression-Syntax-for-Azure-Search

Searching Collection(Edm.String) with oData in .net

Azure search allows for an array of strings to be stored against a record.

You can set up the field either by creating it directly in the azure portal, or creating a property and adding the following attributes.

[IsSearchable, IsFilterable, IsFacetable]
 public string[] Category { get; set; }

The IsFilterable  must be set, otherwise the framework will throw an exception

The .net SDK gives the option to set search parameters when making a query which translates into a HTTP request using the oData syntax.

https://docs.microsoft.com/en-us/rest/api/searchservice/odata-expression-syntax-for-azure-search

The documentation shows how to find a single item in the list, but not multiple options in the same list. To search for options 1 or 2 or 3 in the Category property, the filter will need to be formed as follows

$filter=(category/any(c: c eq '1') or category/any(c: c eq '2') or 
category/any(c: c eq '2'))