%md
Displayed matrix works:
$$ \begin{bmatrix}
    a & b \\ b & c
\end{bmatrix}$$

Non-displayed matrix does not work due to some backslash escaping we do not want touching tex:
$\begin{bmatrix}
    a & b \\ b & c
\end{bmatrix}$

Proof that this is the problem:
$\begin{bmatrix}
    a & b \\\\ b & c
\end{bmatrix}$

%html
Displayed matrix works:
$$ \begin{bmatrix}
    a & b \\ b & c
\end{bmatrix}$$

Non-displayed matrix works fine in html mode:
$\begin{bmatrix}
    a & b \\ b & c
\end{bmatrix}$