AI is maximizing Solar for my EV - Part1

An AI application predicting next day’s solar production for a given solar array and configuring EV charger accordingly.

10 min readMay 20, 2024

Three years ago, I started installing solar panels in my backyard.

Soon enough, I switched from an ICE to a Electric Vehicle (EV), and purchased a Solar enabled EV charger (i.e. a charger able to charge solely from renewable energy)

On a perfect day, my 5kw solar array can deliver up to 30kwh, and I can add up to 150km (~100 miles) to my EV, just from the sun.

Being a solar enthusiast/energy conscious person, I quickly ended up asking myself :

Assuming I need to charge my EV by tomorrow evening, should I rather:

charge overnight from the grid to benefit from off-peak rate ?
or wait and and hope to charge “for free” tomorrow with the sun ?

What if I could predict tomorrow’s solar production ? Surely, this would help me make a better decision, and even automate the charging process.

Here I am talking of predicting my own energy production, taking into consideration panels’ orientation, shading, etc.. This has nothing to do with predicting tomorrow’s weather.

The higher the expected next day’s solar production, the less grid energy I will use to charge the car overnight.

Here comes Solar2ev, an AI application (based on deep neural network), predicting next day’s solar for a given solar array, trained on historical meteorological data.

Real world. End to end. Fully automated.

Solar2ev gets its data from the real world (from a nearby weather station), acts on the real world (EV charger), and covers the entire lifecycle of a deep learning project:

Development: Data acquisition, cleaning, search for best AI model, training, accuracy evaluation.
Deployment: Runs unattended on premises, on low cost Linux Single Board Computer (Nvidia Jetson Nano, Raspberry PI)
Management: With a mobile app based on the Blynk platform.
Evolution once deployed: The AI model is monitored via regular accuracy assessment and continuously improved by retraining at the edge.

Solar2ev is written in Python, and is available on github.

It uses Google’s Tensorflow and Blynk for mobile application development.

Solar2ev comes with a mobile app based on the Blynk platform

How it works.

Every day at sunset, I generate a prediction for the next day’s solar, based on meteorological and solar data from current and previous days.

The prediction is mapped to an amount of grid energy, and the EV charger is then programmed to charge the car overnight with that energy.

The higher the prediction, the lower the energy will be used from the grid at night.

For instance, if I am fully confident that tomorrow’s production will be substantial, I can skip overnight charging and charge tomorrow from the sun.

On the other end, if I think production will be low, I’d better charge overnight.

Of course, if my partner tells me to stop messing around with AI, I also have to charge overnight..but that’s another story…

My EV charger can be configured to only use renewable energy, but of course, it can also charge from the grid.

First thing first: Getting data from the real world.

The input to the deep learning model is a time ordered sequence (of meteorological/solar data). I chose hour as the granularity. There is no need to go finer grain since the weather does not change that fast.

For example: running prediction on Wednesday at 7pm, and using a span of 3 days (Monday to Wednesday) means the sequence has 24 + 24 + 18 = 66 elements. Each element contains hourly meteorological data ( temperature, pressure, etc. ) and solar production.

Real world data is used:

To train the AI model. In that case, I need as much data as possible. However, this will be limited by the age of my solar installation (installed 3 year ago)
To perform the prediction (once per day, at sunset).

Getting real world solar data.

My solar installation is based on Enphase micro-inverters; Enphase offers quite a comprehensive set of reporting capabilities (API to get production day by day, download file for batch historical data).

example of the file downloaded from Enphase (solar production per day):

Date/Time,Energy Produced (Wh)   
10/15/2022,"21,245"  
10/16/2022,"19,492"  
10/17/2022,"17,887"  
10/18/2022,"21,156"

This is a bit more tricky. My requirements are:

I need historical (multi years) data, not forecast.
Data has to be as local as possible (not from 100’s of km/miles away).

I decided to “scrap” a website which provides very local weather measurement (recording station is a few miles from where I live) and yields quite granular information.

“Scrapping” works by parsing the jungle of html code returned by the web server (meant to be rendered by the browser) and to extract the relevant data. It may not be the most elegant type of coding, but it is surprisingly quite simple, albeit dependent on any changes on the web page’s layout, (but for production websites, this is quite infrequent).

Scrapping turns this …

into that:

date,hour,temp,humid,direction,wind,pressure
2021-03-10 00:00:00,0,-3.7,88%,Nord-Nord-Est,3,1020.7
2021-03-10 00:00:00,1,-4.5,89%,Sud-Sud-Ouest,2,1022

Cleaning data.

Real world data is typically messy. It can be incomplete, ambiguous, inconsistent, and cannot be used without cleaning. For instance:

From time to time, the weather website can miss some hourly data. In some instances an entire day may be missing. Those blanks need to be filled.
I built my solar array incrementally, with multiple increments (from initially 1Kw to 5Kw today). I cannot just compare a production from 2021 to a production in 2024.
Over the course of 3 years, the internet went down a couple of times, and solar data could not be reported. Again blanks have to be filled.

Massaging data.

Data such as month, hour and wind direction are “cyclical”. For instance, January is semantically “close” to December, 23h is “close” to 0h.

If we were representing month with numbers, (such as January = 1 and December = 12), this closeness would be lost. However, this closeness is very valuable for the neural network’s learning process.

To alleviate this, cyclical data are represented with sine and cosine, which makes 23h numerically close to 0h.

Finally.

I store the cleaned data into a csv file which becomes the input to the training process.

~25000 row. One row per hour for the life duration of my solar installation.

# feature_input.csv

date,month,hour,temp,humid,direction,wind,pressure,direction_sin,direction_cos,sin_month,cos_month,sin_hour,cos_hour,production
2021-03-19,3,21,-5.1,90.0,3,5,1020.8,0.87,0.5,1.0,0.0,-0.71,0.71,15.25
2021-03-19,3,22,-6.3,91.0,2,8,1021.3,0.64,0.77,1.0,0.0,-0.5,0.87,15.25

All the sheets are cleaned. Let’s analyze the data.

Looking at the data.

Before jumping into model’s training, it is always a good idea to have a look at the data at hand.

Solar production (kWh) per day. From low (blue) to high (red)

The chart above shows the solar production per day. As expected, it is higher in Summer, but can be pretty good from April to Oct.

The chart above shows the distribution of solar data. 50% of the days are above the purple line (i.e. above 17kWh/day), and 50% are below. Likewise, 25% are below 8kWh, and 25% are above 25kWh. (remember those values, we will use them later).

This is the same distribution viewed by month. The dash lines represent the 25/50/75% thresholds described earlier. The vertical “shape” represent the production value for each month.

I was a bit surprised to see how fast the production ramps up from February to March, and reassured to see that the production is decent until October.

I guess the ramp is caused by the orientation and shading of my panels. Moreover, snow on the panels does not help.

The chart above shows how many days (of the past 3 years) had a given production. Quite a few days with very good production (the peak on the right means that when is it sunny, it is very sunny), quite a few days of lousy production (the peak on the left, when is it bad, it is bad). In between, we get all combinations of partly sunny days.

How does solar data relate to meteorological data ?

Note that discovering the response to this question is exactly the purpose of the neural network. Here we just glance at the data at hand, mostly to verify it makes sense.

The chart above shows the production (the color dots) for a given barometric pressure (X axis) and a given temperature (Y axis). There is one dot per day, and the redish/larger the cloud of dots, the higher the production.

Production is indeed quite related to temperature.
Pressure also affects production. When the pressure is low, the production is low. The effect is however not as strong as it is for temperature (the production does not always increases with the pressure).

This makes sense.. temperature is somehow correlated to the number of sun hours per day (both are higher in summer). Likewise, low pressure means cloudy sky.

Another way to look at the relationship between data is to analyze their correlation. It could be positive correlation (both values vary in the same direction) or negative correlation (values vary in opposite direction).

The table above shows such correlations. Just look at the last row to check the relationship between solar production and meteorological data.

The production seems negatively correlated with humidity (when one increases, the other decreases). Wind speed seems more important than wind direction. The production is also strongly negatively correlated with the month (winter versus summer ):

Note: in the table above, month is defined as abs(month — 6). So a high number (12–6, 1–6) is “winter”, ergo the low production. But we knew this already !!

Enough charts, let’s train our model.

Sorry, not quite yet, I need to transform the raw data into input and output features:

input features: what the model will sees as input.
output features: what it will spit out (the prediction).

Input features.

The model will be trained from time ordered sequences (aka time series)

There are many ways I can configure those sequences:

I can set how many days in the past I want to go, whether sequences contain all consecutive hours, or only one hour every two (or 3 or 4.. ).

# 4 days, including today
days_in_seq = 4

# elements within a sequence's are 1 hour apart
sampling = 1 # running prediction at 7pm
today_skipped_hours = 6

Shorter sequences mean “less to remember” for the neural network, but also “less to chew on”. Later, we will try with different parameter values to see what works best.

I can select the actual meteorological data to include in the sequence:

('temp', 'production', 'sin_month', 'cos_month', 'pressure')

I can set the starting point for each sequence:

# sequences used for training starts at those hours.
selection = [0, 1, 2, 3, 4, 5, 6, 7, 8]

selection = [0]: Only sequences starting at 0h will be included. The advantage is that this is exactly the type of sequences the model will see once in operation, but the big drawback is that this only allows for ~1000 sequences (as we have ~1000 days of historical data). This is a very small (much too small) training set.
selection = [0, 1, 2,…, 22, 23]. Sequences starting at every hour will be included. We get a much larger training set (24000 sequences), even though it may be less representative of what will be seen in operation.

All those configuration options allow me to experiment and look at what works best.

Output features.

I actually build 2 models:

One predicting a production value: The model’s output is a number, (eg 13, 6kWh).
One prediction a production class: The model’s output is an interval (eg “between 7.8kWh and 17.3kWh”).

Pedantically speaking, the former is called regression, the later classification.

Why to have 2 models ? I can check later that they “agree”

The number of intervals (aka classes) and their boundaries is configurable:

# categorical = True means classification
# else regression

categorical = True# classes boundaries
# 4 classes"prod_bins":[0, 7.83, 17.37, 25.14, 1000000]

4 classes looks good to me; the first class correspond to a production below 8kWh, and the last to a production > 25kWh.

Hum…we already saw those numbers somewhere. Yes, I remember! in the solar production distribution.

In the data gathered from the real world, 25% of the solar production values are below 7.83kWh whereas 25% are above 25.14kWh.

Using those intervals generates a balanced training set, meaning thatno class is more represented than others.

So if I pick a class at random, there is 25% chance I will pick the right one.

Any model that is correct more than 25% of the time is better than random pick.

Training.

see Part2 (to be published soon) for how to build and train a neural network, and what results did I get