A lot of the documentation that Intel and Arm Technology gives away is specifically of interest to software developers, who these days would be unlikely to be a direct sales lead that Intel or Arm Holdings would want to pursue.
They however contribute to indirect sales, through Arm's and Intel's partners. So there is value to having the information out there.
Vendors like TI and NXP, which are some of the easier ones to get documentation out of, haven't actually changed as much as you would have thought.
Pre-late 90s, early 2000s, whether it was gathered by you calling up a sales office, writing in, visiting a vendor's booth at a trade fair, or sending in the forms that used to commonly be included as the last pages of their printed documentation, the information being collected to ship you the printed material became a sales lead, which someone from the nearest regional sales office would determine if it was worth following up on. (Nowadays, followup is mostly automated email spam, but sales people still call or make site visits to more lucrative leads.)
Since engineers and corporate buyers would fill out that material with office phone numbers and addresses, it wasn't really considered PII at the time, and they wouldn't normally bother retaining information from a hobbyist or student because it wouldn't become a sale.
Chip samples and free or at printing cost documentation being sent to hobbyist was mostly a goodwill thing, since positive experiences made it more likely you would pick them if you went out into industry.
Ironically, that sort of thing started dying out in the late 2000s, early 2010s, because the growth in popularity of hobby electronics happened around the same time vendors started doing major cuts in budgets for their sales department. (It became pretty common that the regional sales office would be closed, and the vendor would have just one sales rep for an entire region. There is also a whole digression into the Arduino-chasing low cost devkits trend that happened at the same time, I could get into.)
Another factor that lead to material being placed behind free account walls, is the US State Department enforcement of export restrictions on dual-use technologies. Vendors (and their sales channels) have legal obligations to ensure that the people receiving access to technical information or dual-use technology aren't on the export restrictions Entity List, or fronting for a person, organization, or country that is on it.
Now companies like Qualcomm are another barrel of fish. They deliberately do not want to deal with customers that are doing low-volume production. By focusing on high-volume OEMs, they can operate with fewer sales/application engineering staff than vendors that support lower volume customers. Modern ICs are not trivial to design into products, and it doesn't take many hours of sales support to make small-volume sales unprofitable for a vendor. (Support for a hobbyist integrating a sampled part is a net negative.) Issues arising from mistakes or oversights during integration by OEMs sometimes get blamed by the public and reviewers on the SoC too, so poor integration can cause PR problems that impact sales. There are also regulatory requirements that come into play, because they specifically focus on the telecoms segment. Code running on or integrating with baseband processors needs to be certified. (Network operators also often require their own interop certification before a device is allowed to be connected to their network.)
Another issue that impacts almost all vendors is that SoCs often contain a lot of licensed IP. PowerVR GPUs were integrated into a lot of SoCs, for example. Encode/decode codecs are often under restrictive IP licenses, and the hardware details of the specialized coprocessors that run them can reveal a lot about implemented specification. Complex peripherals like USB controllers and even interconnects like an internal network on chip fabric are other common licensed third party IP.
Then vendors also still try to do security by obscurity. The ME in Intel processors is one of those classic examples of that.
Another factor is vendor's trying to preserve product segmentation. The use of common chip dies across a product family, and then binning or fusing a particular die for a particular product segment is another reason vendors like to make documentation hard to get. It is sometimes effective in preventing customers from trying to use the questionably good undocumented functionality on cheaper parts instead of buying the more expensive part. It also obfuscates how a particular generic IP block might be getting reused across market segments with specialized firmware blobs to have it function as a dedicated and specialized coprocessor for that market.
In an ideal world, a lot of this stuff wouldn't be an issue, and I'm also frustrated by a lot of this stuff when it comes to my own personal projects too, especially since I'm no longer in the know on the latest stuff, but it is understandable why things are the way they are, even if I find a lot of it very silly.
