Using the framework of the quantum separation of variables (SoV) for higher rank quantum integrable lattice models [1], we introduce some foundations to go beyond the obtained complete transfer matrix spectrum description, and open the way to the computation of matrix elements of local operators. This first amounts to obtain simple expressions for scalar products of the so-called separate states (transfer matrix eigenstates or some simple generalization of them)... In the higher rank case, left and right SoV bases are expected to be pseudo-orthogonal, that is for a given SoV co-vector, there could be more than one non-vanishing overlap with the vectors of the chosen right SoV basis. For simplicity, we describe our method to get these pseudo-orthogonality overlaps in the fundamental representations of the $\mathcal{Y}(gl_3)$ lattice model with $N$ sites, a case of rank 2. The non-zero couplings between the co-vector and vector SoV bases are exactly characterized. While the corresponding SoV-measure stays reasonably simple and of possible practical use, we address the problem of constructing left and right SoV bases which do satisfy standard orthogonality. In our approach, the SoV bases are constructed by using families of conserved charges. This gives us a large freedom in the SoV bases construction, and allows us to look for the choice of a family of conserved charges which leads to orthogonal co-vector/vector SoV bases. We first define such a choice in the case of twist matrices having simple spectrum and zero determinant. Then, we generalize the associated family of conserved charges and orthogonal SoV bases to generic simple spectrum and invertible twist matrices. Under this choice of conserved charges, and of the associated orthogonal SoV bases, the scalar products of separate states simplify considerably and take a form similar to the $\mathcal{Y}(gl_2)$ rank one case. (read more)