K-Maxoids Clustering¶
Example comparing k-means and k-maxoids clustering methods.
K-maxoids automatically preserves extreme periods by selecting points closest to the convex hull.
Author: Maximilian Hoffmann
Import pandas and the relevant time series aggregation class
%load_ext autoreload
%autoreload 2
from pathlib import Path
import pandas as pd
import plotly.express as px
import plotly.io as pio
import tsam
from tsam import ClusterConfig
pio.renderers.default = "notebook_connected"
# Ensure results directory exists
RESULTS_DIR = Path("results")
RESULTS_DIR.mkdir(exist_ok=True)
import warnings
# Added to every example notebook: silence the v3 column-order
# FutureWarning in the rendered docs (tsam v4 returns result columns in
# input order; see migration guide).
warnings.filterwarnings(
"ignore", category=FutureWarning, message=".*sorted alphabetically.*"
)
Input data¶
Read in time series from testdata.csv with pandas
raw = pd.read_csv("testdata.csv", index_col=0)
Show a slice of the dataset
raw.head()
| GHI | T | Wind | Load | |
|---|---|---|---|---|
| 2009-12-31 23:30:00 | 0 | -2.1 | 7.1 | 375.478394 |
| 2010-01-01 00:30:00 | 0 | -2.8 | 8.6 | 364.541326 |
| 2010-01-01 01:30:00 | 0 | -3.3 | 9.7 | 357.416844 |
| 2010-01-01 02:30:00 | 0 | -3.2 | 9.8 | 350.191306 |
| 2010-01-01 03:30:00 | 0 | -3.2 | 9.4 | 345.161449 |
Show the shape of the raw input data: 4 types of timeseries (GHI, Temperature, Wind and Load) for every hour in a year
raw.shape
(8760, 4)
Create a plot function for the temperature for a visual comparison of the time series
# Use tsam.unstack_to_periods() with plotly for heatmap visualization
# px.imshow(unstacked["column"].values.T) creates interactive heatmaps
Plot an example series - in this case the temperature
# Original temperature heatmap
unstacked = tsam.unstack_to_periods(raw, period_duration=24)
px.imshow(
unstacked["T"].values.T,
labels={"x": "Day", "y": "Hour", "color": "Temperature"},
title="Original Temperature",
aspect="auto",
)
Simple k-mean aggregation¶
Initialize an aggregation class object with k-means as method for eight typical days, without any integration of extreme periods. Alternative methods are 'averaging', 'hierarchical', 'kmedoids' and 'kmaxoids'.
result = tsam.aggregate(
raw,
n_clusters=8,
period_duration=24,
cluster=ClusterConfig(method="kmeans"),
)
Create the typical periods
cluster_representatives = result.cluster_representatives
Show shape of typical periods: 4 types of timeseries for 8*24 hours
cluster_representatives.shape
(192, 4)
Save typical periods to .csv file
cluster_representatives.to_csv(RESULTS_DIR / "testperiods_kmeans.csv")
Repredict the original time series based on the typical periods
reconstructed = result.reconstructed
Plot the repredicted data
# K-means predicted temperature heatmap
unstacked_kmeans = tsam.unstack_to_periods(reconstructed, period_duration=24)
px.imshow(
unstacked_kmeans["T"].values.T,
labels={"x": "Day", "y": "Hour", "color": "Temperature"},
title="K-means Predicted Temperature",
aspect="auto",
)
As seen, they days with the minimal temperature are excluded. In case that they are required they can be added to the aggregation as follow.
k-maxoids aggregation including extreme periods¶
Initialize a time series aggregation based on k-maxoids, which automatically searches for points closest to the convex hull.
result_maxoids = tsam.aggregate(
raw,
n_clusters=8,
period_duration=24,
cluster=ClusterConfig(method="kmaxoids"),
preserve_column_means=False,
)
Create the typical periods
cluster_representatives_maxoids = result_maxoids.cluster_representatives
The aggregation can also be evaluated by indicators
result_maxoids.accuracy
AccuracyMetrics( rmse=0.1704 (weighted), mae=0.1187 (weighted), rmse_duration=0.0930 (weighted) )
Repredict the original time series based on the typical periods
reconstructed_maxoids = result_maxoids.reconstructed
Plot repredicted data
# K-maxoids predicted temperature heatmap
unstacked_maxoids = tsam.unstack_to_periods(reconstructed_maxoids, period_duration=24)
px.imshow(
unstacked_maxoids["T"].values.T,
labels={"x": "Day", "y": "Hour", "color": "Temperature"},
title="K-maxoids Predicted Temperature",
aspect="auto",
)
Here bigger biggest values and lower lowest values can be observed compared to k-means clustering.
Comparison of the aggregations¶
It was shown for the temperature, but both times all four time series have been aggregated. Therefore, we compare here also the duration curves of the electrical load for the original time series, the aggregation with k-mean, and the k-maxoids aggregation.
# Duration curve comparison using plotly express
comparison_data = {
"Original": raw,
"8 typ days (Centroids)": reconstructed,
"8 typ days (Maxoids)": reconstructed_maxoids,
}
frames = []
for name, df in comparison_data.items():
sorted_vals = df["Load"].sort_values(ascending=False).reset_index(drop=True)
frames.append(
pd.DataFrame(
{"Hour": range(len(sorted_vals)), "Load": sorted_vals, "Method": name}
)
)
long_df = pd.concat(frames, ignore_index=True)
px.line(
long_df,
x="Hour",
y="Load",
color="Method",
title="Duration Curve Comparison - Load",
)
Or as unsorted time series for an example week
# Time slice comparison using plotly express
frames = []
for name, df in comparison_data.items():
sliced = df.loc["20100210":"20100218", ["Load"]].copy()
sliced["Method"] = name
frames.append(sliced)
long_df = pd.concat(frames).reset_index(names="Time")
px.line(
long_df,
x="Time",
y="Load",
color="Method",
title="Time Slice Comparison - Load (Feb 10-18)",
)