Visualization & Quality Analysis¶

This notebook demonstrates how to analyze the quality of time series aggregation using tsam's built-in plotting tools.

Quick Reference¶

For heatmaps — use tsam.unstack_to_periods() with plotly:

unstacked = tsam.unstack_to_periods(df, period_duration=24)
px.imshow(unstacked["Load"].values.T, labels={"x": "Day", "y": "Hour", "color": "Load"})

Accessor methods (result.plot.*) — for validation after aggregation:

• compare(columns, mode) — Original vs reconstructed ("overlay", "side_by_side", "duration_curve")
• residuals(columns, mode) — Error analysis ("time_series", "histogram", "by_period", "by_timestep")
• cluster_weights() — Bar chart of cluster sizes
• cluster_representatives(columns) — Line plots of typical periods
• cluster_members(columns, clusters, slider) — All original periods per cluster with representative highlighted
• accuracy() — Bar chart of RMSE / MAE / RMSE (Duration) metrics
• segment_durations() — Bar chart of segment lengths (requires segmentation)

Data properties (result.*) — for direct access:

• result.original — Original DataFrame
• result.reconstructed — Reconstructed DataFrame (cached)
• result.residuals — Difference: original − reconstructed
• result.cluster_assignments — Array of cluster indices per period

Table of Contents¶

1. Load and aggregate data
2. Visual comparison (heatmaps, duration curves, line plots)
3. Cluster analysis (weights, representatives)
4. Error analysis (accuracy metrics, residuals)
5. Comparing aggregation configurations
6. Extreme period preservation
7. Segmentation analysis

In [1]:

import pandas as pd
import plotly.express as px
import plotly.io as pio

import tsam
from tsam import ClusterConfig, ExtremeConfig, SegmentConfig

pio.renderers.default = "notebook_connected"
import warnings

# Added to every example notebook: silence the v3 column-order
# FutureWarning in the rendered docs (tsam v4 returns result columns in
# input order; see migration guide).
warnings.filterwarnings(
    "ignore", category=FutureWarning, message=".*sorted alphabetically.*"
)

import pandas as pd import plotly.express as px import plotly.io as pio import tsam from tsam import ClusterConfig, ExtremeConfig, SegmentConfig pio.renderers.default = "notebook_connected" import warnings # Added to every example notebook: silence the v3 column-order # FutureWarning in the rendered docs (tsam v4 returns result columns in # input order; see migration guide). warnings.filterwarnings( "ignore", category=FutureWarning, message=".*sorted alphabetically.*" )

1. Load Data and Run Aggregation¶

In [2]:

# Load test data (8760 hours = 1 year of hourly data)
raw = pd.read_csv("testdata.csv", index_col=0)
print(f"Data shape: {raw.shape}")
print(f"Columns: {list(raw.columns)}")
raw.head()

# Load test data (8760 hours = 1 year of hourly data) raw = pd.read_csv("testdata.csv", index_col=0) print(f"Data shape: {raw.shape}") print(f"Columns: {list(raw.columns)}") raw.head()

Data shape: (8760, 4)
Columns: ['GHI', 'T', 'Wind', 'Load']

Out[2]:

	GHI	T	Wind	Load
2009-12-31 23:30:00	0	-2.1	7.1	375.478394
2010-01-01 00:30:00	0	-2.8	8.6	364.541326
2010-01-01 01:30:00	0	-3.3	9.7	357.416844
2010-01-01 02:30:00	0	-3.2	9.8	350.191306
2010-01-01 03:30:00	0	-3.2	9.4	345.161449

In [3]:

# Run aggregation with 12 typical days
result = tsam.aggregate(
    raw,
    n_clusters=12,
    period_duration=24,
    cluster=ClusterConfig(method="hierarchical"),
)

print(f"Number of clusters: {result.n_clusters}")
print(f"Timesteps per period: {result.n_timesteps_per_period}")
print(f"Total original periods: {len(raw) // result.n_timesteps_per_period}")

# Run aggregation with 12 typical days result = tsam.aggregate( raw, n_clusters=12, period_duration=24, cluster=ClusterConfig(method="hierarchical"), ) print(f"Number of clusters: {result.n_clusters}") print(f"Timesteps per period: {result.n_timesteps_per_period}") print(f"Total original periods: {len(raw) // result.n_timesteps_per_period}")

Number of clusters: 12
Timesteps per period: 24
Total original periods: 365

2. Visual Comparison: Original vs Reconstructed¶

Both the original and the reconstructed data can be visualised as heatmaps or line plots, either of the full time series or of the duration curves.

2.1 Heatmaps¶

These plots show the full year, with periods (days) on the x-axis and timesteps (hours) on the y-axis. The data must be preprocessed using the tsam.unstack_to_periods() method to create these plots.

2.1.1 Plot the Time Series From the Original Data Frame¶

In [4]:

# Reshape raw data for heatmap visualization
unstacked = tsam.unstack_to_periods(raw, period_duration=24)

# Create heatmap with plotly express
px.imshow(
    unstacked["T"].values.T,
    labels={"x": "Day", "y": "Hour", "color": "Temperature"},
    title="Original Temperature",
    aspect="auto",
)

# Reshape raw data for heatmap visualization unstacked = tsam.unstack_to_periods(raw, period_duration=24) # Create heatmap with plotly express px.imshow( unstacked["T"].values.T, labels={"x": "Day", "y": "Hour", "color": "Temperature"}, title="Original Temperature", aspect="auto", )

2.1.2 Plot the Time Series From the Aggregation Result Object¶

The original, unaltered time series can also be accessed from the results.

In [5]:

# Original data heatmap using result.original
unstacked_orig = tsam.unstack_to_periods(result.original, period_duration=24)
px.imshow(
    unstacked_orig["T"].values.T,
    labels={"x": "Day", "y": "Hour", "color": "Temperature"},
    title="Original Temperature (from result)",
    aspect="auto",
)

# Original data heatmap using result.original unstacked_orig = tsam.unstack_to_periods(result.original, period_duration=24) px.imshow( unstacked_orig["T"].values.T, labels={"x": "Day", "y": "Hour", "color": "Temperature"}, title="Original Temperature (from result)", aspect="auto", )

2.1.3 Plot the Time Series From the Aggregation Result Object¶

The results contain the reconstructed time series from the aggregated typical periods. As can be seen, the reconstructed time series deviates slightly from the original time series.

In [6]:

# Reconstructed data heatmap using result.reconstructed
unstacked_recon = tsam.unstack_to_periods(result.reconstructed, period_duration=24)
px.imshow(
    unstacked_recon["T"].values.T,
    labels={"x": "Day", "y": "Hour", "color": "Temperature"},
    title="Reconstructed Temperature",
    aspect="auto",
)

# Reconstructed data heatmap using result.reconstructed unstacked_recon = tsam.unstack_to_periods(result.reconstructed, period_duration=24) px.imshow( unstacked_recon["T"].values.T, labels={"x": "Day", "y": "Hour", "color": "Temperature"}, title="Reconstructed Temperature", aspect="auto", )

2.1.4 Create a Multi Column Plot¶

It is also possible to create a multi-column plot for closely comparing various time series. This is demonstrated here using multiple time series that have been reconstructed.

In [7]:

# Multi-column heatmaps of reconstructed data
for col in ["GHI", "T", "Load"]:
    px.imshow(
        unstacked_recon[col].values.T,
        labels={"x": "Day", "y": "Hour", "color": col},
        title=f"Reconstructed {col}",
        aspect="auto",
    ).show()

# Multi-column heatmaps of reconstructed data for col in ["GHI", "T", "Load"]: px.imshow( unstacked_recon[col].values.T, labels={"x": "Day", "y": "Hour", "color": col}, title=f"Reconstructed {col}", aspect="auto", ).show()

2.1.5 Compare Original and Reconstructed Time Series in a Multicolumn Plot¶

In [8]:

# Compare original vs reconstructed for specific columns
for col in ["T", "Load"]:
    fig_orig = px.imshow(
        unstacked_orig[col].values.T,
        labels={"x": "Day", "y": "Hour", "color": col},
        title=f"Original {col}",
        aspect="auto",
    )
    fig_orig.show()
    fig_recon = px.imshow(
        unstacked_recon[col].values.T,
        labels={"x": "Day", "y": "Hour", "color": col},
        title=f"Reconstructed {col}",
        aspect="auto",
    )
    fig_recon.show()

# Compare original vs reconstructed for specific columns for col in ["T", "Load"]: fig_orig = px.imshow( unstacked_orig[col].values.T, labels={"x": "Day", "y": "Hour", "color": col}, title=f"Original {col}", aspect="auto", ) fig_orig.show() fig_recon = px.imshow( unstacked_recon[col].values.T, labels={"x": "Day", "y": "Hour", "color": col}, title=f"Reconstructed {col}", aspect="auto", ) fig_recon.show()

2.2 Duration Curves¶

Duration curves are created by sorting a time series in descending order. These curves are an important tool for determining how well the aggregation preserves the value distribution.

The most convenient way to analyse the duration curves of the original and reconstructed time series is to use the result.plot.compare() method. It is also demonstrated how to analyse the original and reconstructed duration curves.

2.2.1 Compare Original and Reconstructed Duration Curve¶

In [9]:

# Accessor: Compare original vs reconstructed duration curves
result.plot.compare(mode="duration_curve")

# Accessor: Compare original vs reconstructed duration curves result.plot.compare(mode="duration_curve")

2.2.2 Plot Duration Cruves of the Orignal¶

In [10]:

# Duration curve with plotly express (raw data)
frames = []
for col in ["Load", "GHI"]:
    sorted_vals = raw[col].sort_values(ascending=False).reset_index(drop=True)
    frames.append(
        pd.DataFrame(
            {"Hour": range(len(sorted_vals)), "Value": sorted_vals, "Column": col}
        )
    )
long_df = pd.concat(frames, ignore_index=True)

px.line(long_df, x="Hour", y="Value", color="Column", title="Original Duration Curves")

# Duration curve with plotly express (raw data) frames = [] for col in ["Load", "GHI"]: sorted_vals = raw[col].sort_values(ascending=False).reset_index(drop=True) frames.append( pd.DataFrame( {"Hour": range(len(sorted_vals)), "Value": sorted_vals, "Column": col} ) ) long_df = pd.concat(frames, ignore_index=True) px.line(long_df, x="Hour", y="Value", color="Column", title="Original Duration Curves")

2.2.3 Plot Reconstructed Duration Curves¶

In [11]:

# Duration curves for reconstructed data with plotly express
frames = []
for col in result.reconstructed.columns:
    sorted_vals = (
        result.reconstructed[col].sort_values(ascending=False).reset_index(drop=True)
    )
    frames.append(
        pd.DataFrame(
            {"Hour": range(len(sorted_vals)), "Value": sorted_vals, "Column": col}
        )
    )
long_df = pd.concat(frames, ignore_index=True)

px.line(
    long_df, x="Hour", y="Value", color="Column", title="Reconstructed Duration Curves"
)

# Duration curves for reconstructed data with plotly express frames = [] for col in result.reconstructed.columns: sorted_vals = ( result.reconstructed[col].sort_values(ascending=False).reset_index(drop=True) ) frames.append( pd.DataFrame( {"Hour": range(len(sorted_vals)), "Value": sorted_vals, "Column": col} ) ) long_df = pd.concat(frames, ignore_index=True) px.line( long_df, x="Hour", y="Value", color="Column", title="Reconstructed Duration Curves" )

2.3 Line Plots¶

Compare original vs reconstructed as line plots. Use plotly's interactive zoom/pan to explore specific time periods. The Overlay can be

2.3.1 Compare by Overlay¶

In [12]:

# Accessor: Compare overlay mode (same color per column, dash differentiates Original/Reconstructed)
# Use plotly's interactive zoom to explore specific time ranges
result.plot.compare(
    columns=["T", "Load"],
    mode="overlay",
    title="Temperature and Load Comparison (use zoom to explore)",
)

# Accessor: Compare overlay mode (same color per column, dash differentiates Original/Reconstructed) # Use plotly's interactive zoom to explore specific time ranges result.plot.compare( columns=["T", "Load"], mode="overlay", title="Temperature and Load Comparison (use zoom to explore)", )

2.3.2 Compare Side by Side¶

In [13]:

# Accessor: Compare side-by-side mode
result.plot.compare(
    columns=["GHI"],
    mode="side_by_side",
    title="Solar Irradiance Comparison (side_by_side)",
)

# Accessor: Compare side-by-side mode result.plot.compare( columns=["GHI"], mode="side_by_side", title="Solar Irradiance Comparison (side_by_side)", )