Input Output Manager¶

FINE provides additional functionalities for input/output management. These include plotting functions, saving results in Excel files, storing input and output data in netCDF files, and the exploitation of the optimized energy system.

Standard I/O¶

standardIO ¶

Functions:

getDualValues –

Get dual values of an optimized pyomo instance.
getShadowPrices –

Get dual values of constraint ("shadow prices").
plotLocationalColorMap –

Plot the data of a component for each location.
plotLocations –

Plot locations from a shape file.
plotOperation –

Plot operation time series of a component at a location.
plotOperationColorMap –

Plot operation time series of a component at a location.
plotPieChart –

Plot pie charts on a map.
plotTransmission –

Plot build transmission lines from a shape file.
timer –

Track the time taken by a function (wrapper).
writeOptimizationOutputToExcel –

Write optimization output to an Excel file.

getDualValues ¶

getDualValues(pyM)

Get dual values of an optimized pyomo instance.

:param pyM: optimized pyomo instance :type pyM: pyomo Concrete Model

:return: Pandas Series with dual values

getShadowPrices ¶

getShadowPrices(
    esM,
    constraint,
    ip=0,
    dualValues=None,
    hasTimeSeries=False,
    periodOccurrences=None,
    periodsOrder=None,
)

Get dual values of constraint ("shadow prices").

:param esM: considered energy system model :type esM: EnergySystemModel class instance

:param constraint: constraint from which the dual values should be obtained (e.g. pyM.commodityBalanceConstraint) :type constraint: pyomo.core.base.constraint.SimpleConstraint

:param ip: investment period of transformation path analysis. :type ip: int

:param dualValues: dual values of the optimized model instance. If it is not specified, it is set by using the function getDualValues(). |br| * the default value is None :type dualValues: None or Series

:param hasTimeSeries: If the constaint is time dependent, this parameter concatenates the dual values to a full time series (particularly usefull if time series aggregation was considered). |br| * the default value is False :type hasTimeSeries: bool

:param periodOccurrences: Only required if hasTimeSeries is set to True. |br| * the default value is None :type periodOccurrences: list or None

:param periodsOrder: Only required if hasTimeSeries is set to True. |br| * the default value is None :type periodsOrder: list or None

:return: Pandas Series with the dual values of the specified constraint

plotLocationalColorMap ¶

plotLocationalColorMap(
    esM,
    compName,
    locationsShapeFileName,
    indexColumn,
    ip=0,
    perArea=True,
    areaFactor=1000.0,
    crs="EPSG:3035",
    variableName="capacityVariablesOptimum",
    doSum=False,
    cmap="viridis",
    vmin=0,
    vmax=-1,
    zlabel=None,
    figsize=(6, 6),
    fontsize=12,
    save=False,
    fileName="capacity.png",
    dpi=200,
    **kwargs,
)

Plot the data of a component for each location.

Required arguments:

:param esM: considered energy system model :type esM: EnergySystemModel class instance

:param compName: component name :type compName: string

:param locationsShapeFileName: file name or path to a shape file :type locationsShapeFileName: string

:param indexColumn: name of the column in which the location indices are stored :type indexColumn: string

Default arguments:

:param ip: investment period |br| * the default value is 0 :type ip: int

:param perArea: indicates if the capacity should be given per area |br| * the default value is False :type perArea: boolean

:param areaFactor: meter * areaFactor = km --> areaFactor = 1e3 (--> capacity/km) |br| * the default value is 1e3 :type areaFactor: scalar > 0

:param crs: coordinate reference system |br| * the default value is 'EPSG:3035' :type crs: string

:param variableName: parameter for plotting installed capacity ('_capacityVariablesOptimum') or operation ('_operationVariablesOptimum'). In case of plotting the operation, set the parameter doSum to True. |br| * the default value is '_capacityVariablesOptimum' :type variableName: string

:param doSum: indicates if the variable has to be summarized for the location (e.g. for operation variables) |br| * the default value is False :type doSum: boolean

:param cmap: heat map (color map) (see matplotlib options) |br| * the default value is 'viridis' :type cmap: string

:param vmin: minimum value in heat map |br| * the default value is 0 :type vmin: integer

:param vmax: maximum value in heat map. If -1, vmax is set to the maximum value of the operation time series. |br| * the default value is -1 :type vmax: integer

:param zlabel: z-label of the plot |br| * the default value is 'operation' :type zlabel: string

:param figsize: figure size in inches |br| * the default value is (12,4) :type figsize: tuple of positive floats

:param fontsize: font size of the axis |br| * the default value is 12 :type fontsize: positive float

:param save: indicates if figure should be saved |br| * the default value is False :type save: boolean

:param fileName: output file name |br| * the default value is 'capacity.png' :type fileName: string

:param dpi: resolution in dots per inch |br| * the default value is 200 :type dpi: scalar > 0

plotLocations ¶

plotLocations(
    locationsShapeFileName,
    indexColumn,
    plotLocNames=False,
    crs="EPSG:3035",
    faceColor="none",
    edgeColor="black",
    fig=None,
    ax=None,
    linewidth=0.5,
    figsize=(6, 6),
    fontsize=12,
    save=False,
    fileName="",
    dpi=200,
    **kwargs,
)

Plot locations from a shape file.

Required arguments:

:param locationsShapeFileName: file name or path to a shape file :type locationsShapeFileName: string

:param indexColumn: name of the column in which the location indices are stored :type indexColumn: string

Default arguments:

:param plotLocNames: indicates if the names of the locations should be plotted |br| * the default value is False :type plotLocNames: boolean

:param crs: coordinate reference system |br| * the default value is 'EPSG:3035' :type crs: string

:param faceColor: face color of the plot |br| * the default value is 'none' :type faceColor: string

:param edgeColor: edge color of the plot |br| * the default value is 'black' :type edgeColor: string

:param fig: None or figure to which the plot should be added |br| * the default value is None :type fig: matplotlib Figure

:param ax: None or ax to which the plot should be added |br| * the default value is None :type ax: matplotlib Axis

:param linewidth: linewidth of the plot |br| * the default value is 0.5 :type linewidth: positive float

:param figsize: figure size in inches |br| * the default value is (6,6) :type figsize: tuple of positive floats

:param fontsize: font size of the axis |br| * the default value is 12 :type fontsize: positive float

:param save: indicates if figure should be saved |br| * the default value is False :type save: boolean

:param fileName: output file name |br| * the default value is 'operation.png' :type fileName: string

:param dpi: resolution in dots per inch |br| * the default value is 200 :type dpi: scalar > 0

plotOperation ¶

plotOperation(
    esM,
    compName,
    loc,
    ip=0,
    locTrans=None,
    tMin=0,
    tMax=-1,
    variableName="operationVariablesOptimum",
    xlabel="time step",
    ylabel="operation time series",
    figsize=(12, 4),
    color="k",
    fontsize=12,
    save=False,
    fileName="operation.png",
    dpi=200,
    **kwargs,
)

Plot operation time series of a component at a location.

Required arguments:

:param esM: considered energy system model :type esM: EnergySystemModel class instance

:param compName: component name :type compName: string

:param loc: location :type loc: string

Default arguments:

:param ip: investment period |br| * the default value is 0 :type ip: int

:param locTrans: second location, required when Transmission components are plotted |br| * the default value is None :type locTrans: string

:param tMin: first time step to be plotted (starting from 0) |br| * the default value is 0 :type tMin: integer

:param tMax: last time step to be plotted |br| * the default value is -1 (i.e. the last available index) :type tMax: integer

:param variableName: name of the operation time series. Checkout the component model class to see which options are available. |br| * the default value is '_operationVariablesOptimum' :type variableName: string

:param xlabel: x-label of the plot |br| * the default value is 'time step' :type xlabel: string

:param ylabel: y-label of the plot |br| * the default value is 'operation time series' :type ylabel: string

:param figsize: figure size in inches |br| * the default value is (12,4) :type figsize: tuple of positive floats

:param color: color of the operation line |br| * the default value is 'k' :type color: string

:param fontsize: font size of the axis |br| * the default value is 12 :type fontsize: positive float

:param save: indicates if figure should be saved |br| * the default value is False :type save: boolean

:param fileName: output file name |br| * the default value is 'operation.png' :type fileName: string

:param dpi: resolution in dots per inch |br| * the default value is 200 :type dpi: scalar > 0

plotOperationColorMap ¶

plotOperationColorMap(
    esM,
    compName,
    loc,
    ip=0,
    locTrans=None,
    nbPeriods=365,
    nbTimeStepsPerPeriod=24,
    variableName="operationVariablesOptimum",
    cmap="viridis",
    vmin=0,
    vmax=-1,
    xlabel="period",
    ylabel="timestep per period",
    zlabel="",
    figsize=(12, 4),
    fontsize=12,
    save=False,
    fileName="",
    xticks=None,
    yticks=None,
    xticklabels=None,
    yticklabels=None,
    monthlabels=False,
    dpi=200,
    pad=0.12,
    aspect=15,
    fraction=0.2,
    orientation="horizontal",
    **kwargs,
)

Plot operation time series of a component at a location.

Required arguments:

:param esM: considered energy system model :type esM: EnergySystemModel class instance

:param compName: component name :type compName: string

:param loc: location :type loc: string

Default arguments:

:param ip: investment period of transformation path analysis. :type ip: int

:param locTrans: second location, required when Transmission components are plotted |br| * the default value is None :type locTrans: string

:param nbPeriods: number of periods to be plotted |br| * the default value is 365 :type nbPeriods: integer

:param nbTimeStepsPerPeriod: time steps per period to be plotted (nbPeriods*nbTimeStepsPerPeriod=length of time series) |br| * the default value is 24 :type nbTimeStepsPerPeriod: integer

:param variableName: name of the operation time series. Checkout the component model class to see which options are available. |br| * the default value is '_operationVariablesOptimum' :type variableName: string

:param cmap: heat map (color map) (see matplotlib options) |br| * the default value is 'viridis' :type cmap: string

:param vmin: minimum value in heat map |br| * the default value is 0 :type vmin: integer

:param vmax: maximum value in heat map. If -1, vmax is set to the maximum value of the operation time series. |br| * the default value is -1 :type vmax: integer

:param xlabel: x-label of the plot |br| * the default value is 'day' :type xlabel: string

:param ylabel: y-label of the plot |br| * the default value is 'hour' :type ylabel: string

:param zlabel: z-label of the plot |br| * the default value is 'operation' :type zlabel: string

:param figsize: figure size in inches |br| * the default value is (12,4) :type figsize: tuple of positive floats

:param fontsize: font size of the axis |br| * the default value is 12 :type fontsize: positive float

:param save: indicates if figure should be saved |br| * the default value is False :type save: boolean

:param fileName: output file name |br| * the default value is 'operation.png' :type fileName: string

:param xticks: user specified ticks of the x axis |br| * the default value is None :type xticks: list

:param yticks: user specified ticks of the ý axis |br| * the default value is None :type yticks: list

:param xticklabels: user specified tick labels of the x axis |br| * the default value is None :type xticklabels: list

:param yticklabels: user specified tick labels of the ý axis |br| * the default value is None :type yticklabels: list

:param monthlabels: specifies if month labels are to be plotted (only works correctly if 365 days are specified as the number of periods) |br| * the default value is False :type monthlabels: boolean

:param dpi: resolution in dots per inch |br| * the default value is 200 :type dpi: scalar > 0

:param pad: pad parameter of colorbar |br| * the default value is 0.12 :type pad: float

:param aspect: aspect parameter of colorbar |br| * the default value is 15 :type aspect: float

:param fraction: fraction parameter of colorbar |br| * the default value is 0.2 :type fraction: float

:param orientation: orientation parameter of colorbar |br| * the default value is 'horizontal' :type orientation: float

plotPieChart ¶

plotPieChart(
    locFilePath,
    results_df,
    Property_to_plot="capacity",
    indexColumn_in_shp="index",
    color_list=[
        "skyBlue",
        "green",
        "yellowGreen",
        "#FFB732",
        "yellow",
        "darkOrange",
        "#996300",
        "steelBlue",
        "darkBlue",
    ],
    scaling_factor=500,
    legend_fontsize=14,
)

Plot pie charts on a map.

plotTransmission ¶

plotTransmission(
    esM,
    compName,
    transmissionShapeFileName,
    loc0,
    loc1,
    ip=0,
    crs="EPSG:3035",
    variableName="capacityVariablesOptimum",
    color="k",
    loc=7,
    alpha=0.5,
    ax=None,
    fig=None,
    linewidth=10,
    figsize=(6, 6),
    fontsize=12,
    save=False,
    fileName="",
    dpi=200,
    **kwargs,
)

Plot build transmission lines from a shape file.

Required arguments:

:param esM: considered energy system model :type esM: EnergySystemModel class instance

:param compName: component name :type compName: string

:param transmissionShapeFileName: file name or path to a shape file :type transmissionShapeFileName: string

:param loc0: name of the column in which the location indices are stored (e.g. start/end of line) :type loc0: string

:param loc1: name of the column in which the location indices are stored (e.g. end/start of line) :type loc1: string

Default arguments:

:param ip: investment periods |br| * the default value is 0 :type ip: int

:param crs: coordinate reference system |br| * the default value is 'EPSG:3035' :type crs: string

:param variableName: parameter for plotting installed capacity ('_capacityVariablesOptimum') or operation ('_operationVariablesOptimum'). |br| * the default value is '_capacityVariablesOptimum' :type variableName: string

:param color: color of the transmission line |br| * the default value is 'k' :type color: string

:param loc: location of the legend in the plot |br| * the default value is 7 :type loc: 0 <= integer <= 10

:param alpha: transparency of the legend |br| * the default value is 0.5 :type alpha: 0 <= scalar <= 1

:param fig: None or figure to which the plot should be added |br| * the default value is None :type fig: matplotlib Figure

:param ax: None or ax to which the plot should be added |br| * the default value is None :type ax: matplotlib Axis

:param linewidth: line width of the plot |br| * the default value is 0.5 :type linewidth: positive float

:param figsize: figure size in inches |br| * the default value is (6,6) :type figsize: tuple of positive floats

:param fontsize: font size of the axis |br| * the default value is 12 :type fontsize: positive float

:param save: indicates if figure should be saved |br| * the default value is False :type save: boolean

:param fileName: output file name |br| * the default value is 'operation.png' :type fileName: string

:param dpi: resolution in dots per inch |br| * the default value is 200 :type dpi: scalar > 0

timer ¶

timer(func)

Track the time taken by a function (wrapper).

:param func: Function

.. note:: Usage as a decorator before a function -> @timer

writeOptimizationOutputToExcel ¶

writeOptimizationOutputToExcel(
    esM,
    outputFileName="scenarioOutput",
    optSumOutputLevel=2,
    optValOutputLevel=1,
)

Write optimization output to an Excel file.

:param esM: EnergySystemModel instance in which the optimized model is hold :type esM: EnergySystemModel instance

:param outputFileName: name of the Excel output file (without .xlsx ending) |br| * the default value is 'scenarioOutput' :type outputFileName: string

:param optSumOutputLevel: output level of the optimization summary (see EnergySystemModel). Either an integer (0,1,2) which holds for all model classes or a dictionary with model class names as keys and an integer (0,1,2) for each key (e.g. {'StorageModel':1,'SourceSinkModel':1,...} |br| * the default value is 2 :type optSumOutputLevel: int (0,1,2) or dict

:param optValOutputLevel: output level of the optimal values. Either an integer (0,1) which holds for all model classes or a dictionary with model class names as keys and an integer (0,1) for each key (e.g. {'StorageModel':1,'SourceSinkModel':1,...}

* 0: all values are kept.
* 1: Lines containing only zeroes are dropped.

|br| * the default value is 1

:type optValOutputLevel: int (0,1) or dict

xarray I/O¶

xarrayIO ¶

Functions:

convertDatasetsToEnergySystemModel –

Take dictionary of xarray datasets (with esM instance data) and convert
convertOptimizationInputToDatasets –

Take esM instance input and convert it into xarray datasets.
convertOptimizationOutputToDatasets –

Take esM instance output and convert it into an xarray dataset.
readNetCDFToDatasets –

Read optimization results from grouped netCDF file to dictionary of
readNetCDFtoEnergySystemModel –

Convert netCDF file into an EnergySystemModel instance.
writeDatasetsToNetCDF –

Save dictionary of xarray datasets (with esM instance data) to a netCDF
writeEnergySystemModelToDatasets –

Convert esM instance (input and output) into a xarray dataset.
writeEnergySystemModelToNetCDF –

Write energySystemModel (input and if exists, output) to netCDF file.

convertDatasetsToEnergySystemModel ¶

convertDatasetsToEnergySystemModel(datasets)

Take dictionary of xarray datasets (with esM instance data) and convert it to an esM instance.

:param datasets: The xarray datasets holding all data required to set up an esM instance. :type datasets: Dict[xr.Dataset]

:return: esM - EnergySystemModel instance :rtype: EnergySystemModel instance

convertOptimizationInputToDatasets ¶

convertOptimizationInputToDatasets(
    esM, useProcessedValues=False
)

Take esM instance input and convert it into xarray datasets.

:param esM: EnergySystemModel instance in which the model is held :type esM: EnergySystemModel instance

Default arguments:

:param useProcessedValues: True if the raw values should be over-written by processed values, False otherwise.
    A requirement for perfect-foresight and by extension for spatial and technology aggregations
    |br| * the default value is False
:type useProcessedValues: bool

:return: xr_ds - esM instance data in xarray dataset format :rtype: xarray.dataset

convertOptimizationOutputToDatasets ¶

convertOptimizationOutputToDatasets(
    esM, optSumOutputLevel=0
)

Take esM instance output and convert it into an xarray dataset.

:param esM: EnergySystemModel instance in which the optimized model is held :type esM: EnergySystemModel instance

:param optSumOutputLevel: Output level of the optimization summary (see EnergySystemModel). Either an integer (0,1,2) which holds for all model classes or a dictionary with model class names as keys and an integer (0,1,2) for each key (e.g. {'StorageModel':1,'SourceSinkModel':1,...} |br| * the default value is 2 :type optSumOutputLevel: int (0,1,2) or dict

:return: xr_ds - EnergySystemModel instance output data in xarray dataset format :rtype: xarray.dataset

readNetCDFToDatasets ¶

readNetCDFToDatasets(
    filePath="my_esm.nc", groupPrefix=None, lazy_load=False
)

Read optimization results from grouped netCDF file to dictionary of xr.Datasets.

:param filePath: output file name of netCDF file (can include full path) |br| * the default value is "my_esm.nc" :type filePath: string

:param groupPrefix: if specified, multiple xarray datasets (with esM instance data) are saved to the same netcdf file. The dictionary structure is then {group_prefix}/{group}/{...} instead of {group}/{...} |br| * the default value is None :type groupPrefix: string

:param lazy_load: If True, the data is not loaded into memory until it is accessed. This can be useful for large datasets. Refer to xarray documentation for more information |br| * the default value is False :type lazy_load: boolean

:return: Nested dictionary containing an xr.Dataset with all result values for each component. :rtype: Nested dict

readNetCDFtoEnergySystemModel ¶

readNetCDFtoEnergySystemModel(filePath, groupPrefix=None)

Convert netCDF file into an EnergySystemModel instance.

:param filePath: file name of netCDF file (can include full path) in which the esM data is stored |br| * the default value is "my_esm.nc" :type filePath: string

:return: EnergySystemModel instance :rtype: EnergySystemModel instance

writeDatasetsToNetCDF ¶

writeDatasetsToNetCDF(
    datasets,
    outputFilePath="my_esm.nc",
    removeExisting=False,
    mode="a",
    groupPrefix=None,
)

Save dictionary of xarray datasets (with esM instance data) to a netCDF file.

Required arguments:

:param datasets: The xarray datasets holding all data required to set up an esM instance. :type datasets: Dict[xr.Dataset]

Default arguments:

:param outputFilePath: output file name of the netCDF file (can include full path) |br| * the default value is "my_esm.nc" :type outputFilePath: string

:param removeExisting: indicates if an existing netCDF file should be removed |br| * the default value is False :type removeExisting: boolean

:param mode: Write (‘w’) or append (‘a’) mode.

* If mode=’w’, any existing file at this location will be overwritten.
* If mode=’a’, existing variables will be overwritten.

|br| * the default value is 'a'

:type mode: string

:param groupPrefix: if specified, multiple xarray datasets (with esM instance data) are saved to the same netcdf file. The dictionary structure is then {group_prefix}/{group}/{...} instead of {group}/{...} |br| * the default value is None :type groupPrefix: string

writeEnergySystemModelToDatasets ¶

writeEnergySystemModelToDatasets(
    esM,
    includeShadowPrices=False,
    shadowPriceConstraintStr="commodityBalanceConstraint",
)

Convert esM instance (input and output) into a xarray dataset.

:param esM: EnergySystemModel instance in which the optimized model is held :type esM: EnergySystemModel instance

:param includeShadowPrices: Whether to include shadow prices in the output xarray dataset. |br| * the default value is False :type includeShadowPrices: boolean

:param shadowPriceConstraintStr: The string to identify the constraints for which shadow prices should be included. |br| * the default value is "commodityBalanceConstraint" :type shadowPriceConstraintStr: string

:return: xr_dss_results - esM instance (input and output) data in xarray dataset format :rtype: xr.DataSet

writeEnergySystemModelToNetCDF ¶

writeEnergySystemModelToNetCDF(
    esM,
    outputFilePath="my_esm.nc",
    overwriteExisting=False,
    optSumOutputLevel=0,
    groupPrefix=None,
    includeShadowPrices=False,
    shadowPriceConstraintStr="commodityBalanceConstraint",
)

Write energySystemModel (input and if exists, output) to netCDF file.

:param esM: EnergySystemModel instance in which the model is held :type esM: EnergySystemModel instance

:param outputFilePath: output file name (can include full path) |br| * the default value is "my_esm.nc" :type file_path: string

:param overwriteExisting: Overwrite existing netCDF file |br| * the default value is False :type outputFileName: boolean

:param optSumOutputLevel: Output level of the optimization summary (see EnergySystemModel). Either an integer (0,1,2) which holds for all model classes or a dictionary with model class names as keys and an integer (0,1,2) for each key (e.g. {'StorageModel':1,'SourceSinkModel':1,...} |br| * the default value is 2 :type optSumOutputLevel: int (0,1,2) or dict

:param groupPrefix: if specified, multiple xarray datasets (with esM instance data) are saved to the same netcdf file. The dictionary structure is then {group_prefix}/{group}/{...} instead of {group}/{...} |br| * the default value is None :type group_prefix: string

:param includeShadowPrices: Whether to include shadow prices in the output netCDF file. |br| * the default value is False :type includeShadowPrices: boolean

:param shadowPriceConstraintStr: The string to identify the constraints for which shadow prices should be included. |br| * the default value is "commodityBalanceConstraint" :type shadowPriceConstraintStr: string

:return: Nested dictionary containing xr.Dataset with all result values for each component. :rtype: Dict[str, Dict[str, xr.Dataset]]

Exploit Output¶

dictIO ¶

Functions:

exportToDict –

Write the input arguments of EnergySysteModel and its Components input to a dictionary.
importFromDict –

Convert the dictionaries created by the exportToDict function to an EnergySystemModel.
reconstruct_full_timeseries –

Reconstruct the full timeseries from the time series aggregation (TSA) results.

exportToDict ¶

exportToDict(
    esM, useProcessedValues=False, useTSAvalues=False
)

Write the input arguments of EnergySysteModel and its Components input to a dictionary.

:param esM: EnergySystemModel instance in which the optimization model is held :type esM: EnergySystemModel instance

:return: esmDict, compDict - dicts containing input arguments of EnergySysteModel and its Components input, respectively

importFromDict ¶

importFromDict(esmDict, compDict)

Convert the dictionaries created by the exportToDict function to an EnergySystemModel.

:param esMDict: dictionary created from exportToDict contains all esM information :type dict: dictionary instance

:param compDict: dictionary create from exportToDict containing all component information :type dict: dictionary instance

:return: esM - EnergySystemModel instance in which the optimized model is held

reconstruct_full_timeseries ¶

reconstruct_full_timeseries(esM, timeseries, ip)

Reconstruct the full timeseries from the time series aggregation (TSA) results.