This is part 2 in my series on monitoring my house's energy consumption and production (with solar panels). For the introduction see Part 1 - Introduction. In this part we will take a look at the program that forms the base of the energy consumption meter monitor: the worker service. If you are interested in the source code, it is available on my GitHub (https://github.com/vnbaaij/MeterMonitor). It contains (parts of) other open source projects and in that spirit, i'm making my changes/modifications/additions available to the community again.

Every programming adventure starts with 'File->New->Project' (or a CLI 'dotnet new ...' equivalent) and so does this one. I decided to go with the 'Worker Service' template. As stated in the docs: 'The ASP.NET Core Worker Service template provides a starting point for writing long running service apps' and that is exactly what I need. If you ever need/want/like to do a a program that facilitates a long running process, i would definitely advise you to check out this template. You basically get all the logic needed in just 2 classes; a Worker class that contains the service logic:

public class Worker : BackgroundService
{
    private readonly ILogger<Worker> _logger;

    public Worker(ILogger<Worker> logger)
    {
        _logger = logger;
    }

    protected override async Task ExecuteAsync(CancellationToken stoppingToken)
    {
        while (!stoppingToken.IsCancellationRequested)
        {
            _logger.LogInformation("Worker running at: {time}", DateTimeOffset.Now);
            await Task.Delay(1000, stoppingToken);
        }
    }
}

And a Program class that wires up the service to the host:

public class Program
{
    public static void Main(string[] args)
    {
        CreateHostBuilder(args).Build().Run();
    }

    public static IHostBuilder CreateHostBuilder(string[] args) =>
        Host.CreateDefaultBuilder(args)
            .ConfigureServices((hostContext, services) =>
            {
                services.AddHostedService<Worker>();
            });
}

More information about the worker service can be found here. A host is an object that encapsulates an app's resources, such as Dependency Injection (DI), Logging, Configuration and IHostedService implementations. More information about the generic host can be found here.

A nice bonus you get when using this template is you can turn the program into a real Windows and/or Linux service very easily. To make a Windows service out of this, you add the Microsoft.Extensions.Hosting.WindowsServices NuGet package and add UseWindowsService method to the CreateHostBuilder method with the IHostBuilder fluent API. For Linux you do the same but with the Microsoft.Extensions.Hosting.Systemd package and the UseSystemd() method. You can actually add both and the runtime is smart enough to ignore the one that is not appropriate for the environment you are running the service on. The final CreateHostBuilder would then look like this:

public static IHostBuilder CreateHostBuilder(string[] args) =>
    Host.CreateDefaultBuilder(args)
        .UseSystemd()
        .UseWindowsService()
        .ConfigureServices((hostContext, services) =>
        {
            services.AddHostedService<Worker>();
        });

I'm not using these service extensions yet, but I do make use of other standard .NET Core functionality like Dependency Injection and the configuration framework. My CreateHostBuilder methode looks like this:

public static IHostBuilder CreateHostBuilder(string[] args) =>
    Host.CreateDefaultBuilder(args)
        .ConfigureServices((hostContext, services) =>
        {
            services.Configure<ConfigSettings(hostContext.Configuration.GetSection("Configuration"));
            services.AddTransient<IMeterReader, MeterReader>();
            services.AddHostedService<MeterWorker>();
        });

In the first line of ConfigureServices I register a configuration instance in the service collection (see section below). The second line adds an instance of an IMeterReader implementation in a transient way. This means a new service is created each time it is requested from the service container. For the other two available service lifetime options see 'Service Lifetimes' in the dependency injection section of the ASP.NET Core documentation.The third line adds the worker service to the service container by means of the AddHostedService extension method. I'll discuss the configuration and the MeterWorker in this blog. The MeterReader will be discussed in a later part.

Configuration

Configuration in ASP.NET Core is performed using one or more configuration providers. These providers read configuration data from key-value pairs using a variety of configuration sources like files, Azure KeyVault and command line arguments. Because of using the CreateDefaultBuilder method, the app is all wired up to read configuration options from a appsettings.json file. You can have hierarchical configuration data in that file by using what is called the options pattern. With this pattern you create a simple class with properties that correspond to the keys in the hierarchical file. You can then add that class to the dependency injection service container in the services.Configure call (which is exactly what I am doing).  So by using the appsettings.json

{
  "Configuration": {
    "SerialSettings": {
      "Port": "/dev/ttyUSB0",
      "Baud": 115200,
      "DataBits": 8,
      "Parity": 0,
      "StopBits": 1
    },
    "ReadInterval": "00:00:58",
    "DataFilesPath": "./Data",
    "SaveDataFiles": false,
    "StorageConnectionstring": "xxxx",
    "TablenamePrefix": "meterdata",
  },
  "Logging": {
    "LogLevel": {
      "Default": "Information",
      "Microsoft": "Warning",
      "Microsoft.Hosting.Lifetime": "Warning"
    }
  }
}

and the SerialSettings and ConfigSettings classes

public class SerialSettings
{
    public string Port { get; set; }
    public int Baud { get; set; }
    public int DataBits { get; set; }
    public Parity Parity { get; set; }
    public StopBits StopBits { get; set; }
}

public class ConfigSettings
{
    public SerialSettings SerialSettings { get; set; }
    public TimeSpan ReadInterval { get; set; }
    public string DataFilesPath { get; set; }
    public bool SaveDataFiles { get; set; }
    public string StorageConnectionstring { get; set; }
    public string TablenamePrefix { get; set; }
}

and the DI, I can use the (strongly typed) configuration values where ever I need them. See the code sample below where I inject the ConfigSettings class in the MeterWorker (the worker service class) constructor and then read a connection string from the settings file by just getting a property on the ConfigSettings instance. You'll actually won't find the connection string in the appsettings.json file. I'm using the .NET Secret Manager to store this in a file (UserSecrets.json) outside of the source code folder so this does not get checked in to the Github repository.

public MeterWorker(ILogger<MeterWorker> logger, IMeterReader meterReader, IOptions<ConfigSettings> config)
{
    _logger = logger;
    _meterReader = meterReader;
    _config = config.Value;

    // Retrieve storage account information from connection string.
    _storageTableHelper = new StorageTableHelper(_config.StorageConnectionstring);
}

The worker service

As we can see in the first code snippet, the most important parts of the worker service are the constructor and the ExecuteAsync method. The constructor is already shown above in the configuration paragraph. Let's now take a closer look at the method that actually makes the worker do some work:

protected override async Task ExecuteAsync(CancellationToken stoppingToken)
{
    FileInfo[] files = null;
    int counter = 0;

    _logger.LogInformation($"MeterMonitor executing at: {DateTimeOffset.Now}" );

    GetFilesList(ref files);

    while (!stoppingToken.IsCancellationRequested)
    {
        SetSourceToFiles(files, ref counter);

        _telegram = await _meterReader.ReadAsStreamAsync();

        _logger.LogInformation($"Extracted data from {_meterReader.Source}\n{_telegram.MeterTimestamp}\nConsumption (low/high):\t{_telegram.PowerConsumptionTariff1}/{_telegram.PowerConsumptionTariff2}\nProduction (low/high):\t{_telegram.PowerProductionTariff1}/{_telegram.PowerProductionTariff2}\n");
        _logger.LogInformation($"Calculated CRC: {_telegram.ComputeChecksum()}");

        if (_telegram.ComputeChecksum() != _telegram.CRC)
        {
            _logger.LogError("Telegram not extracted correctly. Calculated CRC not equal to stored CRC ");
        }

        SaveDataToFile();
        SaveDataToStorageTable();

        await Task.Delay(_config.ReadInterval, stoppingToken);
    }
}

When I started building this, I did not have code yet that could read the energy data from the meter. All I had was the DSMR specification (see part 1) and a couple of files with example messages. My starting point was therefore to build a reader that was capable of getting the information out of a set of files. By using an interface, I was able to later swap out the file reader for a 'real' reader. All the code that had to do with the file capabilities is still there. I also added a couple of data files from my own meter to the repository. That way. it is possible to run the program in a completely standalone fashion on your PC. No smart meter needed, no Raspberry Pi needed! To run this way, make sure you put the name of the folder that contains the data files is in the appsettings.json file (DataFilesPath), and  that the AddTransient call in ConfigureServices uses a FileReader instead of a MeterReader: 
services.AddTransient<IMeterReader, MeterReader>(); ==> services.AddTransient<IMeterReader, FileReader>();
If you happen to have a smart meter, a cable and this program running, you can save your own data files by setting the value of SaveDataFiles in the configuration file to true.

The rest of the method is fairly straightforward. It uses a continously running while loop that only stops if a cancelation is requested (elsewhere in the program). In the loop it fetches the next file (if the implemented interface is a FileReader), it then reads the message in a async pattern, saves the message to a file (if so configured), save the message to an Azure Storage Table and then goes to sleep for a specified period to then start the same routine again. In my situation the smart reader emits a data package (also known as a telegram) every minute. I decided to grab each and everyone of them, so I set the interval to 58 seconds. This gives about 1440 (24*60) messages a day. if you don't want or need all messages, you can just set the ReadInterval to a higher value in the configuration.

One more interesting thing in this method is that I built functionality into the reader to generate a CRC over the telegram. The message from the smart meter also comes with a CRC value. By comparing those, I can make sure that the message is read and parsed correctly and that no data was harmed during the processing. More about that in one of the next parts. Hope to see you there!

Comments


Comments are closed