How to create a smart switch for an electric boiler to save money

My electric water heater consumes a lot of electricity, and it does so at times when it is most expensive. This is because I have dynamic prices which change every hour and I shower at an expensive time. And even if a had a high&low tariff, the problem would still exist.

Most people shower at an expensive moment!

There is more than one way to solve this problem, and I prefer one that is technological in nature.

Solution? A smart switch!

Because I already have flexible rates for my electric car, I need to stick to that. This forces the solution to be a smart switch. This switch will only power the electric water heater when electricity is relatively cheap.

The big picture

A computer aware of flexible rates will control the switch powering the water heater

Small computer

The computer has to download the dynamic prices from the internet once a day. This is easiest to do in Python. That makes the cheapest capable device a Raspberry Pi. But any old laptop should also suffice.

In the future, when building more switches, we can have a central device or website maintain the hourly prices for electricity. Then an ESP32 like device could control the switch and make a big improvement on all fronts.

Find rates website

Although the graph of my provider shows me when prices are high or low, it’s a graph and not very useful to a computer. A table would be better, that’s why we need to find a site that has those.

I choose to use the website https://www.entsoe.eu/, it has hourly price data for almost the entire EU (which is useful for me) and provides the data in both forms.

Extracting data

After finding the right table of data, I want to extract the data so I can use it. For this we need the page’s source code.

Source code for the website on the right

This source code might not look very useful, but it is structured and it is textual. That means that we can search it for our data.

Selenium

My first attempt at this was to fetch the page using a web-request in Python, only to not find the data I was looking for. As it turns out, most websites that present data, load it in after the fact using AJAX and similar techniques.

But then there is Selenium, a Python package that lets us automate an actual browser.

Follow the instruction below to install Selenium on the Raspberry Pi. Or find yourself a good tutorial on it depending on your device.

sudo apt-get install chromium-driver
pip3 install selenium

Save webpage to file

from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.common.keys import Keys

# The web adress where we can find the table with prices
entsoe_url = "https://transparency.entsoe.eu/transmission-domain/r2/dayAheadPrices/show?name=&defaultValue=false&viewType=TABLE&areaType=BZN&atch=false&dateTime.dateTime=20.10.2022+00:00|CET|DAY&biddingZone.values=CTY|10YNL----------L!BZN|10YNL----------L&resolution.values=PT60M&dateTime.timezone=CET_CEST&dateTime.timezone_input=CET+(UTC+1)+/+CEST+(UTC+2)"


def retrieve_prices():
    # Create the automated browser object
    browser = webdriver.Chrome()

    # Have it go to the url annd return the entire content of the page
    browser.get(entsoe_url)
    html = browser.page_source

    # Write html content to disk for testing and debugging if needed. 
    with open('content.txt', 'w') as f:
        f.write(html)
        
    # And close the browser
    browser.quit()

if __name__ == '__main__':
    price_list = retrieve_prices()

This short script will create a browser object, open the web-page en retrieve the entire content. This content will then be written to a file called ‘content.txt’.

Now we have the content stored in a file. No need to keep hitting the web-server while we are debugging.

Find the table

The first price in the table is 74.50, so I open ‘content.txt’ and go look for it.

The price appears in in the file, once, which is where the table starts

Because prices are different every day, I can’t use that to find the beginning of the table. We have to look at text in front of the price that looks kinda specific for that part but also returning every day.

DayAheadPricesMongoEntity

This word only start appearing at the start of the table, so if we can first the first one, our table starts there.

To keep things simple, I create a new script to work with the ‘content.txt’ file from here on. Later on I will combine them into the final script.

# Finds a substring in a string and cuts of everything before that occurrence.
def goto(haystack, needle):
    position = haystack.find(needle)

    if position == 0:
        return haystack

    return haystack[position:]

def retrieve_prices():

    # Read the content of the file.
    with open('content.txt', 'r') as f:
        html = f.readlines()
    html = ''.join(html)
    html = html.replace('\n', '')

    # Find the start of the hours column
    html = goto(html, 'DayAheadPricesMongoEntity')
    
    print(html[0:250])
    
    
if __name__ == '__main__':
    price_list = retrieve_prices()

As we can see in the output of the script, we are at the start of the table

Regex

A Regex, short for regular-expression, is a fancy way to describe and look for text.

data-view-detail-link<\/span>"><\/span>(.*?)<span

This is what the text around a prices looks like in regex format. The (.*?) part is a wildcard match where we expect a price to be.

I expanded the script to look for this expression.

import re


# Finds a substring in a string and cuts of everything before that occurrence.
def goto(haystack, needle):
    position = haystack.find(needle)

    if position == 0:
        return haystack

    return haystack[position:]

def retrieve_prices():

    # Read the content of the file.
    with open('content.txt', 'r') as f:
        html = f.readlines()
    html = ''.join(html)
    html = html.replace('\n', '')

    # Find the start of the hours column
    html = goto(html, 'DayAheadPricesMongoEntity')
    
    # Define what we are looking for
    pattern = 'data-view-detail-link<\/span>"&gt;<\/span>(.*?)<span'
    matches = re.findall(pattern, html)

    # Display all the matches
    for m in matches:
        print(m)
    
    # Make sure there are exactly 24 prices
    assert(len(matches) == 24)
    
    
if __name__ == '__main__':
    price_list = retrieve_prices()

With the regular expression added, the script now finds exactly 24 prices

Make a switch decision

Now we have prices, we can actually make a decision to turn the switch on or off. I added to the second script, the function now returns a list of prices and the main function makes the decision.

    # Return as numbers
    return [float(i) for i in matches]
    
    
if __name__ == '__main__':
    old_day = -1
    old_hour = -1
    
    prices = None
    
    # Run forever
    while True:
        now = datetime.datetime.now()
        
        # New day? Download new prices
        if now.day != old_day:
            prices = retrieve_prices()
            old_day = now.day
            
        # New hour? Check if price is below average
        if now.hour != old_hour:
            average_price = sum(prices) / len(prices)
            if prices[now.hour] < average_price:
                print('Switch on')
            else:
                print('Switch off')    
            old_hour = now.hour
            
        # Sleep for a while  
        time.sleep(10)

Using the prices and current time of day, the program can now make a decision

The logic of the script is very simple. New day? Then download new prices. New hour? Then check if current price is above or below average and switch accordingly. These conditions will be checked every 10 seconds, forever.

Output signal

Now I have create a third script, that does nothing but flip a digital output pin on the Raspberry Pi on and off every 5 seconds. This will be very useful during the building of the hardware part of the switch.

import time
import RPi.GPIO as GPIO

switch_pin = 40    # GPIO21

GPIO.setmode(GPIO.BOARD)
GPIO.setup(switch_pin, GPIO.OUT)

switch = False

while True:
    time.sleep(5)
    switch = not switch
    
    if switch:
        GPIO.output(switch_pin, GPIO.HIGH)
        print('On')
    else:
        GPIO.output(switch_pin, GPIO.LOW)
        print('Off')

Small note about the script. switch = not switch does not mean I doubt its existence. It basically means, if its false, then make it true and the other way around. This makes it go On-Off-On-Off-etc.

The digital pin goes HIGH or LOW every 5 seconds

That is a lot of wire just to measure the voltage, but it does the job.

How to drive the relay

The IO pins on the Raspberry are not powerful enough to drive the relay directly, so I need a man in the middle.

Use a NPN transistor to drive the low side of the relay

I created this circuit on a breadboard and hooked it up between the Raspberry and the relay switch.

The breadboard version of the circuit, and it is actually working

This circuit actually works, and with this heavy relay I can switch loads up to 40 amps. That is more than enough to switch my water heater.

Now that I saw this working, I feel confident enough to transfer the circuit over to some proto-board.

With the proto-board finished, the smart switch is nearing completion

For the first version of a product, the size is very acceptable to me.

Putting it all together

I took a piece of board and hot-glued all the pieces to it

I went to the store and got myself two extension-cords. One I kept intact to power the Raspberry and the second one is the switched one. I worked open the cable and cut one of the power wires. Then I soldered the ends before I screwed them into the relay.

With the Raspberry still loaded with the testing script, the light went on and off with an interval of a few seconds.

The smart switch is now actually switching main power

Finishing the software

The three separate scripts can be lumped together fairly easy, all the difficult parts are there to re-use. One thing has to be added to the script however. The URL we used for testing was a static one, with a fixed date in it. I added some code to make the URL always for today’s date.

Github

https://github.com/energy-automation/smart-switch

Testing the switch in the shed first

I could now use the switch for my water heater, but that is two floors up in my house and out of sight. I think a better option would be to test the switch for a few days in my shed, switching a lamp, just to see how it manages over time. Just in case it creates a fire or worse, causing the water-heater to give me cold water while in the shower.

Auto-start the script upon reboot

Working on it

Conclusion

Working on it

Places to go from here

Working on it

Update 1

I am working on a friendlier version of the smart-switch. The full size Raspberry Pi is now replaces with a Banana Pi Zero, which is a smaller and has a good antenna. I also made a nice case for the switch.