Given $$\begin{bmatrix} 3{ x }^{ 2 }+10xy & +5{ y }^{ 2 } \end{bmatrix}=\begin{bmatrix} x & y \end{bmatrix}A\begin{bmatrix} x \\ y \end{bmatrix}$$
We will check using options
If option A is the matrix A, then consider RHS
$$\begin{bmatrix} x & y \end{bmatrix}\begin{bmatrix} 3 & 10 \\ 10 & 5 \end{bmatrix}\begin{bmatrix} x \\ y \end{bmatrix}$$
$$=\begin{bmatrix} x & y \end{bmatrix}\begin{bmatrix} 3x+10y \\ 10x+5y \end{bmatrix}$$
$$=\begin{bmatrix} 3{ x }^{ 2 }+20xy & +5{ y }^{ 2 } \end{bmatrix}$$
which is not equal to given LHS
Now, option B, so let $$A=\begin{bmatrix} 10 & 3 \\ 5 & 10 \end{bmatrix}$$
$$\begin{bmatrix} x & y \end{bmatrix}\begin{bmatrix} 10 & 3 \\ 5 & 10 \end{bmatrix}\begin{bmatrix} x \\ y \end{bmatrix}$$
$$=\begin{bmatrix} x & y \end{bmatrix}\begin{bmatrix} 10x+3y \\ 5x+10y \end{bmatrix}$$
$$=\begin{bmatrix} 3{ x }^{ 2 }+8xy & +5{ y }^{ 2 } \end{bmatrix}$$
which is not equal to given LHS
Now, we will try option C
Let $$A=\begin{bmatrix} 3 & -5 \\ -5 & 5 \end{bmatrix}$$
$$\begin{bmatrix} x & y \end{bmatrix}\begin{bmatrix} 3 & -5 \\ -5 & 5 \end{bmatrix}\begin{bmatrix} x \\ y \end{bmatrix}$$
$$=\begin{bmatrix} x & y \end{bmatrix}\begin{bmatrix} 3x-5y \\ -5x+5y \end{bmatrix}$$
$$=\begin{bmatrix} 3{ x }^{ 2 }-10xy & +5{ y }^{ 2 } \end{bmatrix}$$
which is not equal to given LHS
Now, lastly we will try option D.
Let $$A=\begin{bmatrix} 3 & 5 \\ 5 & 5 \end{bmatrix}$$
$$\begin{bmatrix} x & y \end{bmatrix}\begin{bmatrix} 3 & 5 \\ 5 & 5 \end{bmatrix}\begin{bmatrix} x \\ y \end{bmatrix}$$
$$=\begin{bmatrix} x & y \end{bmatrix}\begin{bmatrix} 3x+5y \\ 5x+5y \end{bmatrix}$$
$$=\begin{bmatrix} 3{ x }^{ 2 }+10xy & +5{ y }^{ 2 } \end{bmatrix}$$
which is equal to given LHS