Curious to know which one(s) I'd recommend as your first entry?
No? Well too bad! As a true Touhou Expert™ (not really) here are three worth considering!
Touhou 7 - Perfect Cherry Blossom
A classic all-rounder!
The second Touhou game for PC, it's a game that sticks to the core of the bullet hell genre with great execution. The unique mechanic for this one gives you a temporary shield from time to time, as you collect falling cherry blossom petals during gameplay. You need to think about it during gameplay, and it can be really helpful in a pinch!
With fun fights, simpler mechanics, and generous 1-ups, it's a great entry point that's more forgiving than usual for the series. And the music is incredible!
Touhou 18 - Unconnected Marketeers
A modern romp with a roguelite twist!
This is the latest mainline release! As such, it's available through Steam, and has less risk of incompatibility oddities on modern PCs. That being said, you'll still want to patch a fan translation if you don't understand Japanese!
The game is quite unique compared to the classic core, but I love a lot of the changes made by Unconnected Marketeers. It has a kind of roguelite element to it, where you get to purchase an upgrade after every level. After your first time purchasing that upgrade, you can also select it as a starting upgrade for all future runs, giving it the most replayability of any game in the series. Plus, it makes you feel like you progressed even when you game over as a result!
Upgrades range from movement, survivability, additional attacks, etc. They give runs a unique flavor, and if you'd prefer to stick to the base gameplay, you can go for the hearts/bombs for sale after every stage instead.
This is more subjective, but I also think the backgrounds and bullet patterns in Unconnected Marketeers are some of the prettiest in the series. All around, I think it's an amazing entry -- plus the music is incredible!
Touhou 8 - Imperishable Night
Everyone's Here! Ready for Unexpected Team-Ups?!
Regardless of whether you start with Perfect Cherry Blossom or Unconnected Marketeers, I suggest this one as your follow-up! Imperishable Night is terrific, but it's also got a kind of victory lap feeling to it.
What do I mean by this? Well, for starters, it features unusual pairings of usual main characters working alongside bosses from previous games. It's got a lot of effort put into it, with the largest playable cast, two different final bosses, additional Last Word challenges, etc. It's Touhou at its most Season Finale.
Despite loving the game to bits, I recommend playing another game first because this one might get you bad habits! The unique mechanic in Imperishable Night rewards you for sticking to either unfocused shots or focused shots for a while instead of adapting, giving you an additional continue if you were consistent enough during a stage.
So while this is a little on the weirder side, the game as a whole is a blast, its stage 4 in particular has yet to be surpassed by any other game in the series IMO. Add the sheer amount of content on top, and you have a real winner. Oh, and the music is incredible!
No matter which one you end up playing, I highly recommend using thcrap to translate the game to your display language of choice. You can download it here!
And that's all I got! If you end giving the series a shot I want to hear how it went!! 
