How to add Tag and Category Clouds to a blog running ButterCMS

by Patrick Lee on 04 May 2019 in categories tech with tags ASP.NET Core AzureFunctions AzureTables ButterCMS

Popular self-service blog platforms like WordPress allow to you add Tag Clouds via adding what they call a widget to a sidebar.

I wanted to add one to this site (which is a standard ASP.NET Core 2.2 webapp, but modified to use ButterCMS as a blog backend) and after checking with the team at ButterCMS that a facility to do this didn't already exist, I decided to have a go at creating one.  This is the result.

A word cloud is a useful rough visualisation of the relative frequency of particular words, either within an article, or in the case of a Tag Cloud for a blog, how many articles a particular tag has been added to.  Tags which are more frequently used are shown in larger font than tags which are less frequently used, and each word in the cloud has a hyperlink taking you to the articles related to that tag.

Here's how relative frequencies can be calculated, and then saved to an Azure Table using the existing ButterCMS API (Application Programming Interface) methods (perhaps in future ButterCMS might add APIs to produce this information more directly):

Azure Function (to retrieve the necessary information and update an Azure Storage Table with the latest information once a day)

// Date        Who Comments 
// 04 May 2019 PJL Created to cache the information needed for Tag Clouds and Category Clouds from a ButterCMS blog (to avoid slowing down the website)
//                 At the moment, this function updates the cache table daily
// 04 May 2019 PJL Version 1.0 of FunctionApp1.UpdateTables
// **************************************************************

using ButterCMS;
using Microsoft.Azure.WebJobs;
using Microsoft.Extensions.Logging;
using Microsoft.WindowsAzure.Storage; // added for CloudStorageAccount
using Microsoft.WindowsAzure.Storage.Table; // added for Table
using System;
using System.Collections.Generic;
using System.Linq;
using System.Threading.Tasks;

namespace FunctionApp1
{
    public static class UpdateTables
    {
        [FunctionName("UpdateTables")]
        // NB [TimerTrigger("0 0 0 */1 * *")] means every 1 day
        public async static Task Run([TimerTrigger("0 0 0 */1 * *")]TimerInfo myTimer, ILogger log)
        {
            var apiToken = GetEnvironmentVariable("ButterCMSApiKey");
            var client = new ButterCMSClient(apiToken);
            var tagCloudRecords = new List<WordCloudRecord>();
            var response = await client.ListTagsAsync(false);
            var tags = response;
            foreach (var item in tags)
            {
                var page = 1;
                var postsResponse = await client.ListPostsAsync(page, 10, true, null, null, item.Slug);
                var posts = postsResponse.Data;
                tagCloudRecords.Add(new WordCloudRecord { PartitionKey = "Tag", RowKey = item.Name, count = posts.Count(), slug = item.Slug });
            }
            SetRelativeFrequenciesOfAListOfCloudRecords(tagCloudRecords);

            var categoryCloudRecords = new List<WordCloudRecord>();
            var response2 = await client.ListCategoriesAsync(false);
            var categories = response2;
            foreach (var item in categories)
            {
                var itemResponse = await client.RetrieveCategoryAsync(item.Slug, false);
                var page = 1;
                var postsResponse = await client.ListPostsAsync(page, 10, true, null, item.Slug);
                var posts = postsResponse.Data;
                categoryCloudRecords.Add(new WordCloudRecord { PartitionKey = "Category", RowKey = item.Name, count = posts.Count(), slug = item.Slug });
            }
            SetRelativeFrequenciesOfAListOfCloudRecords(categoryCloudRecords);

            var connectionString = GetEnvironmentVariable("NameForYourAzureStorageAccountSetting");
            var storageAccount = CloudStorageAccount.Parse(connectionString);
            var tableClient = storageAccount.CreateCloudTableClient();
            var tableName = "NameOfYourWordCloudsTable";
            var table = tableClient.GetTableReference(tableName);

            var types = new List<string> { "Tag", "Category" };
            foreach (var type in types)
                await DeleteRecordsOfThisType(table, type);

            // add the new records to the table for each type. NB each record in a batch operation must have the same partition key
            await AddBatchOfRecordsToTable(tagCloudRecords, table);
            await AddBatchOfRecordsToTable(categoryCloudRecords, table);

            log.LogInformation($"C# Timer trigger function executed at: {DateTime.Now}");
        }

        private static void SetRelativeFrequenciesOfAListOfCloudRecords(List<WordCloudRecord> records)
        {
            int totalNumber = (from i in records
                               select i.count).Sum();
            foreach (var item in records)
            {
                if (totalNumber != 0)
                    item.frequency = (double)item.count / (double)totalNumber;
                else
                    item.frequency = 0;
            }
        }

        private static async Task DeleteRecordsOfThisType(CloudTable table, string type)
        {
            TableQuery<WordCloudRecord> query = new TableQuery<WordCloudRecord>().Where(TableQuery.GenerateFilterCondition("PartitionKey", QueryComparisons.Equal, type));
            TableContinuationToken token = null;
            do
            {
                TableQuerySegment<WordCloudRecord> resultSegment = await table.ExecuteQuerySegmentedAsync(query, token);
                token = resultSegment.ContinuationToken;
                foreach (WordCloudRecord entity1 in resultSegment.Results)
                {
                    TableOperation deleteOperation = TableOperation.Delete(entity1);
                    // Execute the operation.
                    await table.ExecuteAsync(deleteOperation);
                }
            } while (token != null);
        }

        private static async Task AddBatchOfRecordsToTable(List<WordCloudRecord> records, CloudTable table)
        {
            TableBatchOperation batchOperation = new TableBatchOperation();
            foreach (WordCloudRecord item in records)
                batchOperation.Insert(item);
            // Execute the batch operation.
            await table.ExecuteBatchAsync(batchOperation);
        }

        private static string GetEnvironmentVariable(string name)
        {
            return
                Environment.GetEnvironmentVariable(name, EnvironmentVariableTarget.Process);
        }
    }

    public class WordCloudRecord : TableEntity
    {
        public WordCloudRecord(string type, string name)
        {
            this.PartitionKey = type;
            this.RowKey = name;
        }

        public WordCloudRecord() { }

        public int count { get; set; }
        public double frequency { get; set; }
        public string slug { get; set; }
    }
}

What the above code does is (within an Azure Function version 2 triggered by a timer once a day):

  1. Use the ButterCMS API to retrieve a list of Tags, and for each tag extract the number of blog posts which are tagged with that tag.
  2. For each tag, it populates a WordCloudRecord (a record to be stored in the Azure Table) with all the necessary fields except the (relative) frequency.
  3. It then calculates the relative frequency for each tag and adds this to each record in the list of tag records (tagCloudRecords).
  4. It does the same for Categories, to produce a list of category records (categoryCloudRecords).

At this point, the code could be used within the Home and or Blog controllers of the website but this would mean that the calls in steps 1 to 4 will be made on each page that you want to display the tag or category clouds on, which could significantly slow down your site's responsiveness. 

So instead, the above Azure Function runs once a day to extract the information and store it within an Azure Storage Table. This should give reasonably accurate tag and category clouds, unless you are publish a very large number of posts on a particular day compared to the total number of posts so far (or if you create new tags or categories during the day).  But the next day the clouds will be up to date again.

Your website can then retrieve the cached information from the Azure table for each view where you want to display the tag and/or category clouds.

For example within an MVC controller where you want the Home/Index and Home/ShowClouds view to show the tag and category clouds (but you could instead put the clouds in a sidebar):

Example MVC Controller: HomeController.cs

using ButterCMS; // added for ButterCMSClient
using Microsoft.AspNetCore.Mvc;
using Microsoft.Extensions.Configuration; // added for IConfiguration
using Microsoft.WindowsAzure.Storage; // added for CloudStorageAccount
using Microsoft.WindowsAzure.Storage.Table; // added for Table
using Project.Models;
using System.Collections.Generic;
using System.Diagnostics;
using System.Threading.Tasks;

namespace Project.Controllers
{
    public class HomeController : Controller
    {
        private ButterCMSClient client;
        private readonly string _connectionString;

        public HomeController(IConfiguration configuration)
        {
            var apiToken = configuration["ButterCMSApiKey"];
            client = new ButterCMSClient(apiToken);
            _connectionString = configuration["NameForYourAzureStorageAccountSetting"];
        }

        public async Task<ActionResult> Index()
        {
            return View();
        }

        public async Task<ActionResult> ShowClouds()
        {
            await GetTagAndCategoryCloudRecords();
            return View();
        }

        private async Task GetTagAndCategoryCloudRecords()
        {
            var storageAccount = CloudStorageAccount.Parse(_connectionString);
            var tableClient = storageAccount.CreateCloudTableClient();
            var tableName = "NameOfYourWordCloudsTable";
            var table = tableClient.GetTableReference(tableName);

            var tagCloudRecords = await ObtainListOfRecordsOfGivenType(table, "Tag");
            var categoryCloudRecords = await ObtainListOfRecordsOfGivenType(table, "Category");
            ViewBag.tagCloudRecords = tagCloudRecords;
            ViewBag.categoryCloudRecords = categoryCloudRecords;
            ViewBag.ShowSideBar = true;
        }

        private static async Task<List<WordCloudRecord>> ObtainListOfRecordsOfGivenType(CloudTable table, string type)
        {
            List<WordCloudRecord> list = new List<WordCloudRecord>();
            TableQuery<WordCloudRecord> query = new TableQuery<WordCloudRecord>().Where(TableQuery.GenerateFilterCondition("PartitionKey", QueryComparisons.Equal, type));
            TableContinuationToken token = null;
            do
            {
                TableQuerySegment<WordCloudRecord> resultSegment = await table.ExecuteQuerySegmentedAsync(query, token);
                token = resultSegment.ContinuationToken;

                foreach (WordCloudRecord entity in resultSegment.Results)
                    list.Add(entity);
            } while (token != null);
            return list;
        }

        // other views and methods ...
    }
}

Example View Page: ShowClouds.cshtml

@using Project.Models;

@{
    ViewData["Title"] = "Show Tag and Category Clouds";
}
<h1>@ViewData["Title"]</h1>

@{
    double minFontSize = 8;
    double maxFontSize = 22;
    List<WordCloudRecord> tagCloudRecords = new List<WordCloudRecord>();
    double largestFrequency = 1;
    double lowestFrequency = 1;
    if (ViewBag.tagCloudRecords != null)
    {
        tagCloudRecords = ViewBag.tagCloudRecords;
        largestFrequency = (from i in tagCloudRecords
                            select i.frequency).Max();
        lowestFrequency = (from i in tagCloudRecords
                           select i.frequency).Min();
    }
    List<WordCloudRecord> categoryCloudRecords = new List<WordCloudRecord>();
    double largestFrequency2 = 1;
    double lowestFrequency2 = 1;
    if (ViewBag.categoryCloudRecords != null)
    {
        categoryCloudRecords = ViewBag.categoryCloudRecords;
        largestFrequency2 = (from i in categoryCloudRecords
                             select i.frequency).Max();
        lowestFrequency2 = (from i in categoryCloudRecords
                            select i.frequency).Min();
    }
}

<div class="container mt-3">
    <h2>
        Category cloud
    </h2>
    <div class="media border p-3">
        <div class="media-body">
            @foreach (var item in categoryCloudRecords)
            {
                var fontSize = minFontSize;
                if (largestFrequency2 != lowestFrequency2)
                {
                    fontSize = minFontSize + (item.frequency - lowestFrequency2) / (largestFrequency2 - lowestFrequency2) * (maxFontSize - minFontSize);
                }
                string fontSizeAsString = fontSize + "pt;";
                <a href="/blog/category/@Uri.EscapeDataString(item.slug)" style="font-size:@fontSizeAsString">@item.RowKey</a>
            }
        </div>
    </div>
</div>

<div class="container mt-3">
    <h2>
        Tag cloud
    </h2>
    <div class="media border p-3">
        <div class="media-body">
            @foreach (var item in tagCloudRecords)
            {
                var fontSize = minFontSize;
                if (largestFrequency != lowestFrequency)
                {
                    fontSize = minFontSize + (item.frequency - lowestFrequency) / (largestFrequency - lowestFrequency) * (maxFontSize - minFontSize);
                }
                string fontSizeAsString = fontSize + "pt;";
                <a href="/blog/tag/@Uri.EscapeDataString(item.slug)" style="font-size:@fontSizeAsString">@item.RowKey</a>
            }
        </div>
    </div>
</div>

The results

And this is what the clouds look like in a sidebar:

ButterCMSWordCloudsShownInSidebar.jpg