Note

You can download this example as a Jupyter notebook or start it in interactive mode.

Creating Expressions

In this notebook, we look at different options to create expressions. A strong focus will be set on the array-like operations: Since variables are represented in array-like structure, we benefit from a lot of well-knwon functionalities which we know from numpy, pandas or xarray.

These are for example

• arithmetic operations to create expressions

• broadcasting to combine smaller and larger arrays

• .loc to select a subset of the original array using indexes

• .where to select where the variable or expression should be active or not

• .shift to shift the whole array along one dimension

• .groupby to group by a key and apply operations on the groups

• .rolling to perform a rolling operation and perform operations

Hint

Nearly all of the functions and properties, that can be accessed from a Variable, can be accesses from a LinearExpression and QuadraticExpression.

Let’s start by creating a model.

import linopy
import pandas as pd
import xarray as xr

time = pd.Index(range(10), name='time')
port = pd.Index(list('abcd'), name='port')

m = linopy.Model()
x = m.add_variables(lower=0, coords=[time], name='x')
y = m.add_variables(lower=0, coords=[time, port], name='y')
m

Linopy LP model
===============

Variables:
----------
 * x (time)
 * y (time, port)

Constraints:
------------
<empty>

Status:
-------
initialized

Arithmetic Operations

Arithmetic operations such as addition (+), subtraction (-), multiplication (*) can be used directly on the variables and expressions in Linopy. These operations are applied element-wise on the variables.

For example, if you want to create a new combined expr z that is the sum of x and y, you can do so as follows:

z = x + y
z

LinearExpression (time: 10, port: 4):
-------------------------------------
[0, a]: +1 x[0] + 1 y[0, a]
[0, b]: +1 x[0] + 1 y[0, b]
[0, c]: +1 x[0] + 1 y[0, c]
[0, d]: +1 x[0] + 1 y[0, d]
[1, a]: +1 x[1] + 1 y[1, a]
[1, b]: +1 x[1] + 1 y[1, b]
[1, c]: +1 x[1] + 1 y[1, c]
                ...
[8, b]: +1 x[8] + 1 y[8, b]
[8, c]: +1 x[8] + 1 y[8, c]
[8, d]: +1 x[8] + 1 y[8, d]
[9, a]: +1 x[9] + 1 y[9, a]
[9, b]: +1 x[9] + 1 y[9, b]
[9, c]: +1 x[9] + 1 y[9, c]
[9, d]: +1 x[9] + 1 y[9, d]

Note

In the addition, the variable x is broadcasted and the return value has the same set of dimensions as y.

Similarly, you can subtract y from x or multiply x and y as follows:

z = x - y
z

LinearExpression (time: 10, port: 4):
-------------------------------------
[0, a]: +1 x[0] - 1 y[0, a]
[0, b]: +1 x[0] - 1 y[0, b]
[0, c]: +1 x[0] - 1 y[0, c]
[0, d]: +1 x[0] - 1 y[0, d]
[1, a]: +1 x[1] - 1 y[1, a]
[1, b]: +1 x[1] - 1 y[1, b]
[1, c]: +1 x[1] - 1 y[1, c]
                ...
[8, b]: +1 x[8] - 1 y[8, b]
[8, c]: +1 x[8] - 1 y[8, c]
[8, d]: +1 x[8] - 1 y[8, d]
[9, a]: +1 x[9] - 1 y[9, a]
[9, b]: +1 x[9] - 1 y[9, b]
[9, c]: +1 x[9] - 1 y[9, c]
[9, d]: +1 x[9] - 1 y[9, d]

z = x * y
z

QuadraticExpression (time: 10, port: 4):
----------------------------------------
[0, a]: +1 x[0] y[0, a]
[0, b]: +1 x[0] y[0, b]
[0, c]: +1 x[0] y[0, c]
[0, d]: +1 x[0] y[0, d]
[1, a]: +1 x[1] y[1, a]
[1, b]: +1 x[1] y[1, b]
[1, c]: +1 x[1] y[1, c]
                ...
[8, b]: +1 x[8] y[8, b]
[8, c]: +1 x[8] y[8, c]
[8, d]: +1 x[8] y[8, d]
[9, a]: +1 x[9] y[9, a]
[9, b]: +1 x[9] y[9, b]
[9, c]: +1 x[9] y[9, c]
[9, d]: +1 x[9] y[9, d]

In all cases, the returned shape is the same. Note that, the output type of the multiplication is a QuadraticExpression and not a LinearExpression.

The z expression, which carries along x and y, has different attributes such as coord_dims, dims, size.

z.coord_dims

('time', 'port')

Important

When combining variables or expression with dimensions of the same name and size, the first object will determine the coordinates of the resulting expression. For example:

other_time = pd.Index(range(10, 20), name='time')
b = m.add_variables(coords=[other_time], name='b')
b

Variable (time: 10)
-------------------
[10]: b[10] ∈ [-inf, inf]
[11]: b[11] ∈ [-inf, inf]
[12]: b[12] ∈ [-inf, inf]
[13]: b[13] ∈ [-inf, inf]
[14]: b[14] ∈ [-inf, inf]
[15]: b[15] ∈ [-inf, inf]
[16]: b[16] ∈ [-inf, inf]
[17]: b[17] ∈ [-inf, inf]
[18]: b[18] ∈ [-inf, inf]
[19]: b[19] ∈ [-inf, inf]

b has the same shape as x, but they have different coordinates. When we combine x and b the coordinates on dimension time will be taken from the first object and the coordinates of the subsequent object will be ignored:

x + b

LinearExpression (time: 10):
----------------------------
[0]: +1 x[0] + 1 b[10]
[1]: +1 x[1] + 1 b[11]
[2]: +1 x[2] + 1 b[12]
[3]: +1 x[3] + 1 b[13]
[4]: +1 x[4] + 1 b[14]
[5]: +1 x[5] + 1 b[15]
[6]: +1 x[6] + 1 b[16]
[7]: +1 x[7] + 1 b[17]
[8]: +1 x[8] + 1 b[18]
[9]: +1 x[9] + 1 b[19]

Using .loc to select a subset

The .loc function allows you to select a subset of the array using indexes. This is useful when you want to apply operations to a specific subset of your variables.

For example, if you want to apply a summation to the variables x and y only for the first 5 time steps, you can do so as follows:

x.loc[:5]

Variable (time: 6)
------------------
[0]: x[0] ∈ [0, inf]
[1]: x[1] ∈ [0, inf]
[2]: x[2] ∈ [0, inf]
[3]: x[3] ∈ [0, inf]
[4]: x[4] ∈ [0, inf]
[5]: x[5] ∈ [0, inf]

x.loc[:5] + y.loc[:5]

LinearExpression (time: 6, port: 4):
------------------------------------
[0, a]: +1 x[0] + 1 y[0, a]
[0, b]: +1 x[0] + 1 y[0, b]
[0, c]: +1 x[0] + 1 y[0, c]
[0, d]: +1 x[0] + 1 y[0, d]
[1, a]: +1 x[1] + 1 y[1, a]
[1, b]: +1 x[1] + 1 y[1, b]
[1, c]: +1 x[1] + 1 y[1, c]
                ...
[4, b]: +1 x[4] + 1 y[4, b]
[4, c]: +1 x[4] + 1 y[4, c]
[4, d]: +1 x[4] + 1 y[4, d]
[5, a]: +1 x[5] + 1 y[5, a]
[5, b]: +1 x[5] + 1 y[5, b]
[5, c]: +1 x[5] + 1 y[5, c]
[5, d]: +1 x[5] + 1 y[5, d]

which is the same as

expr = x + y
expr.loc[:5]

LinearExpression (time: 6, port: 4):
------------------------------------
[0, a]: +1 x[0] + 1 y[0, a]
[0, b]: +1 x[0] + 1 y[0, b]
[0, c]: +1 x[0] + 1 y[0, c]
[0, d]: +1 x[0] + 1 y[0, d]
[1, a]: +1 x[1] + 1 y[1, a]
[1, b]: +1 x[1] + 1 y[1, b]
[1, c]: +1 x[1] + 1 y[1, c]
                ...
[4, b]: +1 x[4] + 1 y[4, b]
[4, c]: +1 x[4] + 1 y[4, c]
[4, d]: +1 x[4] + 1 y[4, d]
[5, a]: +1 x[5] + 1 y[5, a]
[5, b]: +1 x[5] + 1 y[5, b]
[5, c]: +1 x[5] + 1 y[5, c]
[5, d]: +1 x[5] + 1 y[5, d]

In combination with the overwrite of the coordinates, this is useful when you need to combine different selections, like

x.loc[:4] + y.loc[5:]

LinearExpression (time: 5, port: 4):
------------------------------------
[0, a]: +1 x[0] + 1 y[5, a]
[0, b]: +1 x[0] + 1 y[5, b]
[0, c]: +1 x[0] + 1 y[5, c]
[0, d]: +1 x[0] + 1 y[5, d]
[1, a]: +1 x[1] + 1 y[6, a]
[1, b]: +1 x[1] + 1 y[6, b]
[1, c]: +1 x[1] + 1 y[6, c]
                ...
[3, b]: +1 x[3] + 1 y[8, b]
[3, c]: +1 x[3] + 1 y[8, c]
[3, d]: +1 x[3] + 1 y[8, d]
[4, a]: +1 x[4] + 1 y[9, a]
[4, b]: +1 x[4] + 1 y[9, b]
[4, c]: +1 x[4] + 1 y[9, c]
[4, d]: +1 x[4] + 1 y[9, d]

Using .where to select active variables or expressions

The .where function allows you to select where the variable or expression should be active or not. This is useful when you want to apply constraints or operations only to a specific subset of your variables based on a condition. It is quite similar to the functionality of masking, that we showed earlier.

For example, if you want to create an sum of the variables x and y where time is greater than 2, you can do so as follows:

mask = xr.DataArray(time > 2, coords=[time])
(x + y).where(mask)

LinearExpression (time: 10, port: 4):
-------------------------------------
[0, a]: None
[0, b]: None
[0, c]: None
[0, d]: None
[1, a]: None
[1, b]: None
[1, c]: None
                ...
[8, b]: +1 x[8] + 1 y[8, b]
[8, c]: +1 x[8] + 1 y[8, c]
[8, d]: +1 x[8] + 1 y[8, d]
[9, a]: +1 x[9] + 1 y[9, a]
[9, b]: +1 x[9] + 1 y[9, b]
[9, c]: +1 x[9] + 1 y[9, c]
[9, d]: +1 x[9] + 1 y[9, d]

We can use this to make a conditional summation:

(x + y).where(mask) + xr.DataArray(5, coords=[time]).where(~mask, 0)

LinearExpression (time: 10, port: 4):
-------------------------------------
[0, a]: +5
[0, b]: +5
[0, c]: +5
[0, d]: +5
[1, a]: +5
[1, b]: +5
[1, c]: +5
                ...
[8, b]: +1 x[8] + 1 y[8, b]
[8, c]: +1 x[8] + 1 y[8, c]
[8, d]: +1 x[8] + 1 y[8, d]
[9, a]: +1 x[9] + 1 y[9, a]
[9, b]: +1 x[9] + 1 y[9, b]
[9, c]: +1 x[9] + 1 y[9, c]
[9, d]: +1 x[9] + 1 y[9, d]

Using .shift to shift the Variable along one dimension

The .shift function allows you to shift the whole array along one dimension. This is useful when you want to apply constraints or operations that involve a time delay or a shift in the time steps.

For example, if you want to apply a constraint that involves a one time step delay in the variables x and y, you can do so as follows:

y - y.shift(time=1)

LinearExpression (time: 10, port: 4):
-------------------------------------
[0, a]: +1 y[0, a]
[0, b]: +1 y[0, b]
[0, c]: +1 y[0, c]
[0, d]: +1 y[0, d]
[1, a]: +1 y[1, a] - 1 y[0, a]
[1, b]: +1 y[1, b] - 1 y[0, b]
[1, c]: +1 y[1, c] - 1 y[0, c]
                ...
[8, b]: +1 y[8, b] - 1 y[7, b]
[8, c]: +1 y[8, c] - 1 y[7, c]
[8, d]: +1 y[8, d] - 1 y[7, d]
[9, a]: +1 y[9, a] - 1 y[8, a]
[9, b]: +1 y[9, b] - 1 y[8, b]
[9, c]: +1 y[9, c] - 1 y[8, c]
[9, d]: +1 y[9, d] - 1 y[8, d]

Using .groupby to group by a key and apply operations on the groups

The .groupby function allows you to group by a key and apply operations on the groups. This is useful when you want to apply constraints or operations that involve a grouping of the time steps or any other dimension.

For example, if you want to apply a constraint that involves the sum of x and y over every two time steps, you can do so as follows:

group_key = pd.Series(time.values // 2, index=time)
(x + y).groupby(group_key).sum()

LinearExpression (port: 4, group: 5):
-------------------------------------
[a, 0]: +1 x[0] + 1 x[1] + 1 y[0, a] + 1 y[1, a]
[a, 1]: +1 x[2] + 1 x[3] + 1 y[2, a] + 1 y[3, a]
[a, 2]: +1 x[4] + 1 x[5] + 1 y[4, a] + 1 y[5, a]
[a, 3]: +1 x[6] + 1 x[7] + 1 y[6, a] + 1 y[7, a]
[a, 4]: +1 x[8] + 1 x[9] + 1 y[8, a] + 1 y[9, a]
[b, 0]: +1 x[0] + 1 x[1] + 1 y[0, b] + 1 y[1, b]
[b, 1]: +1 x[2] + 1 x[3] + 1 y[2, b] + 1 y[3, b]
                ...
[c, 3]: +1 x[6] + 1 x[7] + 1 y[6, c] + 1 y[7, c]
[c, 4]: +1 x[8] + 1 x[9] + 1 y[8, c] + 1 y[9, c]
[d, 0]: +1 x[0] + 1 x[1] + 1 y[0, d] + 1 y[1, d]
[d, 1]: +1 x[2] + 1 x[3] + 1 y[2, d] + 1 y[3, d]
[d, 2]: +1 x[4] + 1 x[5] + 1 y[4, d] + 1 y[5, d]
[d, 3]: +1 x[6] + 1 x[7] + 1 y[6, d] + 1 y[7, d]
[d, 4]: +1 x[8] + 1 x[9] + 1 y[8, d] + 1 y[9, d]

Using .rolling to perform a rolling operation

The .rolling function allows you to perform a rolling operation and apply operations. This is useful when you want to apply constraints or operations that involve a rolling window of the time steps or any other dimension.

For example, if you want to apply a constraint that involves the sum of x over a rolling window of 3 time steps, you can do so as follows:

x.rolling(time=3).sum()

LinearExpression (time: 10):
----------------------------
[0]: +1 x[0]
[1]: +1 x[0] + 1 x[1]
[2]: +1 x[0] + 1 x[1] + 1 x[2]
[3]: +1 x[1] + 1 x[2] + 1 x[3]
[4]: +1 x[2] + 1 x[3] + 1 x[4]
[5]: +1 x[3] + 1 x[4] + 1 x[5]
[6]: +1 x[4] + 1 x[5] + 1 x[6]
[7]: +1 x[5] + 1 x[6] + 1 x[7]
[8]: +1 x[6] + 1 x[7] + 1 x[8]
[9]: +1 x[7] + 1 x[8] + 1 x[9]