One thing I've heard for ages is that it used to be an assumed part of a proper gentleman's education (say, around the year 1870 or so) that the gentleman would learn to draw - not that every educated gentleman would be ready to a career as a portrait artist, but basic competence with a pencil and paper, to where the assumed standard was that people could give a basic impression of what they had seen.
How was that taught? For mathematics, or literature, or Latin or anything else that young gentlemen were expected to learn, we can find textbooks and instruction manuals, at the college level or at younger levels for use in grammar schools, etc. If we look at those compared to what's in use today, it's remarkable how little has changed. (I actually collect elementary math texts from 1880-1920, and the word problems are surprisingly similar to what I remember)
Drawing though is no longer a skill that schools think everyone should, or even can learn. As far as I can tell, the current practice as far as basic drawing goes is that some people have talent, most people don't, and the only thing you can do is get children to practice and then encourage those who show talent to keep drawing.
Imagine if we tried to teach mathematics that way: just get everyone to practice with, I don't know, cuisenaire rods or something, and the kids who demonstrate mathematical talent are then left alone with a copy of Principia Mathematica and we see what happens. We might get the occasional prodigy who'd do fine, but we certainly wouldn't end up with most people being able to do basic algebra.
So it seems obvious to me that the current attitude towards teaching specifically observational drawing (as opposed to creating artists in general) of just assuming that some people "have it" and some don't (and that therefore the only two choices are "give up" or "just practice more") isn't the only possible way to look at it.
I also think that we should be able to find textbooks or primers or lecture notes or something that talks about how young men of breeding were taught to sketch accurately, but I've never come across a book like that in my crawling through vintage bookstores; it's possible that I don't know the right thing to look for.