Learning goals:

Know the formulas for saturating functions, de-saturating functions, increasing sigmoid functions, and decreasing sigmoid functions.
Know what the parameters in these functions represent, especially in relation to steepness.

A “saturating” function

It increases at first, but only up to a maximum (saturation) level.

Parameters:

$m =$ the saturation level, the value that $f(X)$ approaches as $X$ gets large (think $m$ for maximum)
$h =$ the half-saturation point, the $X$ value at which $f(X) = \frac{1}{2}m$

In [1]:

xmax=15
m = 7
h = 2
f(X) = m * X / (h + X)
p = plot(f(X), (X, 0, xmax), thickness=2)
p += text(r"$f(X) = m \cdot \frac{X}{h + X}$", (xmax/2, 8), color="black", fontsize=18)
p += line(((0, m), (xmax, m)), color="red", linestyle="dashed")
p += text(r"$m$", (0.2, m + 0.3), color="red", fontsize="larger", horizontal_alignment="left")
p += line(((0, m/2), (h, m/2), (h, 0)), color="green", linestyle="dotted")
p += text(r"$X = h$", (h + 0.1, 0.4), color="green", fontsize="larger", horizontal_alignment="left")
p += text(r"$\frac{m}{2}$", (0.2, m/2 + 0.5), color="green", fontsize="larger", horizontal_alignment="left")
p.show(ymin=0, ymax=10, axes_labels=("$X$", "$f(X)$"), figsize=5)

p += text("Saturating function", (xmax/2, 10), color="black", fontsize=12, axes_labels=("$X$", ""))
saturating_plot = p

In [2]:

# An interactive allowing you to manipulate the values of m and h:
@interact
def saturating(
        m=slider(1, 8, 1, default=1, label="$m$"),
        h=slider(1, 10, 1, default=1, label="$h$"),
        show_m=checkbox(False, label="Show $m$"),
        show_h=checkbox(False, label="Show $h$"),
):
    xmax=15
    f(X) = m * X / (h + X)
    label = r"$f(X) = %s \cdot \frac{X}{%s + X}$" % (m, h)
    p = plot(f(X), (X, 0, xmax), thickness=2)
    p += text(label, (xmax/2, 9), color="black", fontsize=18)
    if show_m:
        p += line(((0, m), (xmax, m)), color="red", linestyle="dashed")
        p += text(r"$m$", (0.2, m + 0.3), color="red", fontsize="larger", horizontal_alignment="left")
    if show_h:
        p += line(((0, m/2), (h, m/2), (h, 0)), color="green", linestyle="dotted")
        p += text(r"$X = h$", (h + 0.1, 0.4), color="green", fontsize="larger", horizontal_alignment="left")
        p += text(r"$\frac{m}{2}$", (0.2, m/2 + 0.5), color="green", fontsize="larger", horizontal_alignment="left")
    p.show(ymin=0, ymax=10, axes_labels=("$X$", "$f(X)$"))

Variation on the above: a sigmoid function

It has an “S” shape (sort of), which is where the term “sigmoid” comes from. It also increases only up to a maximum (saturation) level, but it starts out flat (horizontal) for low $X$ values, then curves upward, before leveling off.

Parameters:

$m =$ the saturation level, the value that $f(X)$ approaches as $X$ gets large (think $m$ for maximum)
$h =$ the half-saturation point, the $X$ value at which $f(X) = \frac{1}{2}m$
$n =$ the exponent, which controls the steepness of the transition from low values to high values

In [3]:

xmax=15
m = 7
h = 2
n = 6
f(X) = m * X^n / (h^n + X^n)
p = plot(f(X), (X, 0, xmax), thickness=2)
p += text(r"$f(X) = m \cdot \frac{X^n}{h^n + X^n}$", (xmax/2, 8.5), color="black", fontsize=18)
p += text(r"(Here $n = {}$)".format(n), ((h + xmax)/2, 4), color="black", fontsize=14)
p += line(((0, m), (xmax, m)), color="red", linestyle="dashed")
p += text(r"$m$", (0.2, m + 0.3), color="red", fontsize="larger", horizontal_alignment="left")
p += line(((0, m/2), (h, m/2), (h, 0)), color="green", linestyle="dotted")
p += text(r"$X = h$", (h + 0.1, 0.4), color="green", fontsize="larger", horizontal_alignment="left")
p += text(r"$\frac{m}{2}$", (0.2, m/2 + 0.5), color="green", fontsize="larger", horizontal_alignment="left")
p.show(ymin=0, ymax=10, axes_labels=("$X$", "$f(X)$"), figsize=4.5)

p += text("Increasing sigmoid function", (xmax/2, 10), color="black", fontsize=12, axes_labels=("$X$", ""))
incsigmoid_plot = p

In [4]:

# An interactive allowing you to manipulate the values of n, m and h:
@interact
def sigmoid(
        n=slider(2, 20, 1, default=2, label="$n$"),
        m=slider(1, 8, 1, default=1, label="$m$"),
        h=slider(1, 10, 1, default=1, label="$h$"),
        show_m=checkbox(False, label="Show $m$"),
        show_h=checkbox(False, label="Show $h$"),
):
    xmax=15
    f(X) = m * X^n / (h^n + X^n)
    label = r"$f(X) = %s \cdot \frac{X^{%s}}{%s + X^{%s}}$" % (m, n, h if n == 1 or h == 1 else "%s^{%s}" % (h, n), n)
    p = plot(f(X), (X, 0, xmax), thickness=2)
    p += text(label, (xmax/2, 9), color="black", fontsize=18)
    if show_m:
        p += line(((0, m), (xmax, m)), color="red", linestyle="dashed")
        p += text(r"$m$", (0.2, m + 0.3), color="red", fontsize="larger", horizontal_alignment="left")
    if show_h:
        p += line(((0, m/2), (h, m/2), (h, 0)), color="green", linestyle="dotted")
        p += text(r"$X = h$", (h + 0.1, 0.4), color="green", fontsize="larger", horizontal_alignment="left")
        p += text(r"$\frac{m}{2}$", (0.2, m/2 + 0.5), color="green", fontsize="larger", horizontal_alignment="left")
    p.show(ymin=0, ymax=10, axes_labels=("$X$", "$f(X)$"))

Note: Often, we just use $h = 1$ , so that the function simplifies to the following:

In [16]:

@interact
def sigmoid_simple(
        n=slider(2, 40, 1, default=2, label="$n$"), 
        show_m=checkbox(False, label="Show $m$"), 
):
    xmax=3
    m = 1
    f(X) = m * X^n / (1 + X^n)
    p = plot(f(X), (X, 0, xmax), thickness=2)
    p += text(r"$f(X) = m \cdot \frac{X^{%s}}{1 + X^{%s}}$" % (n,n), (xmax/2, 1.25), color="black", fontsize=18)
    if show_m:
        p += line(((0, m), (xmax, m)), color="red", linestyle="dashed")
    p.show(ymin=0, ymax=1.5, axes_labels=("$X$", "$f(X)$"))

A “de-saturating” function

It decreases, from its initial (maximum) level, down to near zero.

Parameters:

$m =$ the initial (maximum) level, the value of $f(X)$ at $X = 0$
$h =$ the half-saturation point, the $X$ value at which $f(X) = \frac{1}{2}m$

In [17]:

xmax=15
m = 7
h = 2
f(X) = m * h / (h + X)
p = plot(f(X), (X, 0, xmax), thickness=2)
p += text(r"$f(X) = m \cdot \frac{h}{h + X}$", (xmax/2, 8), color="black", fontsize=18)
p += line(((0, m), (1, m)), color="red", linestyle="dashed")
p += text(r"$m$", (0.2, m + 0.3), color="red", fontsize="larger", horizontal_alignment="left")
p += line(((0, m/2), (h, m/2), (h, 0)), color="green", linestyle="dotted")
p += text(r"$X = h$", (h + 0.1, 0.4), color="green", fontsize="larger", horizontal_alignment="left")
p += text(r"$\frac{m}{2}$", (0.2, m/2 + 0.5), color="green", fontsize="larger", horizontal_alignment="left")
p.show(ymin=0, ymax=10, axes_labels=("$X$", "$f(X)$"), figsize=5)

p += text("De-saturating function", (xmax/2, 10), color="black", fontsize=12, axes_labels=("$X$", ""))
desaturating_plot = p

In [18]:

# An interactive allowing you to manipulate the values of m and h:
@interact
def desaturating(
        m=slider(1, 8, 1, default=1, label="$m$"),
        h=slider(1, 10, 1, default=1, label="$h$"),
        show_h=checkbox(False, label="Show $h$"),
):
    xmax=15
    f(X) = m * h / (h + X)
    label = r"$f(X) = %s \cdot \frac{%s}{%s + X}$" % (m, h, h)
    p = plot(f(X), (X, 0, xmax), thickness=2)
    p += text(label, (xmax/2, 9), color="black", fontsize=18)
    p += line(((0, m), (1, m)), color="red", linestyle="dashed")
    p += text(r"$m$", (0.2, m + 0.3), color="red", fontsize="larger", horizontal_alignment="left")
    if show_h:
        p += line(((0, m/2), (h, m/2), (h, 0)), color="green", linestyle="dotted")
        p += text(r"$X = h$", (h + 0.05, 0.4), color="green", fontsize="larger", horizontal_alignment="left")
        p += text(r"$\frac{m}{2}$", (0.1, m/2 + 0.5), color="green", fontsize="larger", horizontal_alignment="left")
    p.show(ymin=0, ymax=10, axes_labels=("$X$", "$f(X)$"))

A decreasing sigmoid function

This one has a backwards “S” shape. Like the de-saturating function, it decreases from its initial (maximum) level down to near zero. However, this one starts out flat (horizontal) for low $X$ values, then curves downward, before leveling off.

Parameters:

$m =$ the initial (maximum) level, the value of $f(X)$ at $X = 0$
$h =$ the half-saturation point, the $X$ value at which $f(X) = \frac{1}{2}m$
$n =$ the exponent, which controls the steepness of the transition from low values to high values

In [19]:

xmax=15
m = 7
h = 2
n = 6
f(X) = m * h^n / (h^n + X^n)
p = plot(f(X), (X, 0, xmax), thickness=2)
p += text(r"$f(X) = m \cdot \frac{h^n}{h^n + X^n}$", (xmax/2, 8), color="black", fontsize=18)
p += text(r"(Here $n = {}$)".format(n), ((h + xmax)/2, 4), color="black", fontsize=14)
p += line(((0, m), (h, m)), color="red", linestyle="dashed")
p += text(r"$m$", (0.2, m + 0.3), color="red", fontsize="larger", horizontal_alignment="left")
p += line(((0, m/2), (h, m/2), (h, 0)), color="green", linestyle="dotted")
p += text(r"$X = h$", (h + 0.1, 0.4), color="green", fontsize="larger", horizontal_alignment="left")
p += text(r"$\frac{m}{2}$", (0.2, m/2 + 0.5), color="green", fontsize="larger", horizontal_alignment="left")
p.show(ymin=0, ymax=10, axes_labels=("$X$", "$f(X)$"), figsize=4.5)

p += text("Decreasing sigmoid function", (xmax/2, 10), color="black", fontsize=12, axes_labels=("$X$", ""))
decsigmoid_plot = p

In [20]:

# An interactive allowing you to manipulate the values of n, m and h:
@interact
def decreasing_sigmoid(
        n=slider(2, 20, 1, default=2, label="$n$"),
        m=slider(1, 8, 1, default=1, label="$m$"),
        h=slider(1, 10, 1, default=1, label="$h$"),
        show_h=checkbox(False, label="Show $h$"),
):
    xmax=15
    f(X) = m * h^n / (h^n + X^n)
    h_tothe_n = h if n == 1 or h == 1 else "%s^{%s}" % (h, n)
    label = r"$f(X) = %s \cdot \frac{%s}{%s + X^{%s}}$" % (m, h_tothe_n, h_tothe_n, n)
    p = plot(f(X), (X, 0, xmax), thickness=2)
    p += text(label, (xmax/2, 9), color="black", fontsize=18)
    p += text(r"$m$", (0.2, m + 0.4), color="red", fontsize="larger", horizontal_alignment="left")
    if show_h:
        p += line(((0, m/2), (h, m/2), (h, 0)), color="green", linestyle="dotted")
        p += text(r"$X = h$", (h - 0.1, 0.4), color="green", fontsize="larger", horizontal_alignment="right")
        p += text(r"$\frac{m}{2}$", (0.2, m/2 + 0.5), color="green", fontsize="larger", horizontal_alignment="left")
    p.show(ymin=0, ymax=10, axes_labels=("$X$", "$f(X)$"))

Note: Often, we just use $h = 1$ , so that the function simplifies to the following:

In [21]:

@interact
def decreasing_sigmoid_simple(
        n=slider(2, 40, 1, default=2, label="$n$"), 
):
    xmax=3
    m = 1
    f(X) = m * 1 / (1 + X^n)
    label = r"$f(X) = \frac{1}{1 + X^{%s}}$" % (n, )
    p = plot(f(X), (X, 0, xmax), thickness=2)
    p += text(label, (xmax/2, 1.25), color="black", fontsize=18)
    p += text(r"$m$", (0.1, m + 0.05), color="red", fontsize="larger", horizontal_alignment="left")
    p.show(ymin=0, ymax=1.5, axes_labels=("$X$", "$f(X)$"))

Summary:

In [22]:

p = graphics_array(((saturating_plot, incsigmoid_plot), (desaturating_plot, decsigmoid_plot)))
p.show(figsize=8)