I love tactical RPGs, but many of them are clunky games that never play as well as they read. This mostly about D&D 4e and Lancer as games which I've played a lot of, and which struggle in places, and I'll point to Blood Neon and Mythic Space as lesser-known games which are advancing the state of the art.
I’m guided by a few pithy quotes.
“Games are a series of interesting decisions.”
–Sid Meier
“In war, everything is very simple. But the simplest things are very hard.”
–Clausewitz
And one mental model for tactics, John Boyd's OODA Loop. The Wikipedia article is a good start, and for further reading I recommend Hammond's The Mind of War for the hagiography, and Hankin's Flying Camelot for the truth. The key thing to keep in mind about Boyd and the OODA Loop is that he was a fighter pilot, his arena was one of split-second decisions in a fluid environment where a moment's inattention could be fatal, and where the word "disoriented" would typically be followed by "controlled flight into terrain", aka Lt. Dumbass iced himself by flying into a mountain. But even if the true applicable of OODA is slippery to grasp, I think the idea holds some use.

Observe
Scouting and its counterpart ambush are fundamental to military tactics. The best way to stay alive is not to be seen (you may quote Monty Python now). Tactical RPGs typically take place on a grid with full knowledge of both sides. Mechanics around hiding and tactical sneaking are notably clunky, requiring either off-grid bookkeeping or very good deliberate ignorance. A second level of observation beyond the presence of the enemy is their description, either figure, image, or text, and what that implies about their tactical capabilities. And a third level is their actual stats. A great tactical RPG should introduce information as a key level of its mechanics, without introducing obfuscation which makes the game harder to play.
Orient
Orientation is the heart of Boyd's model, our ability to translate the raw sensory impressions of observations into meaningful facts about the world. Becoming 'oriented' to the game's model is part of what learning and enjoying the game is about. One thing which many games fail at is respecting working memory, the idea that people can only hold about seven pieces of information in their head. I recall my D&D4e group, with more advanced degrees than people around the table, having trouble managing their own abilities, let alone the synergistic abilities of party as a whole. As orientation is the key to Boyd's model, it's also the key to a great game experience. Building orientation another way to say that you've mastered a game, and it also relates to that fraught word "immersion", as we shift from a character orientation to a player orientation, or find facts which are hard to square with the shape of the shared fiction.
Decide
Once you know what's up, you have to know what to do. Professional military types are big on fast decision cycles, with "getting inside your opponents OODA loop" as the premier goal, such that by the time the enemy is reacting to what you're doing, you've finished and are already doing something else. Decision paralysis is a problem for some players, with too many tightly coupled ramified decisions to work through, but I've found that another problem arises, when the right decision is so obvious that it's no longer interesting. Lancer falls into this trap, where the deep mech building subgame results in a tactical situation that is "build a big stick, use it as frequently as you can." Great tactical games allow moments of tactical brilliance, without requiring brilliance at every turn.
Act
Action is key in war, but one area where I think a lot of games fail is that they don't respect how much time it takes to actually do a turn, in a Taylorist time-motion study sense. And even if people are entirely on the ball and always know exactly what they're going to do, the mechanics of playing out a turn drag. Let's break down a turn from the dragon game.
- Pick up d20
- Roll d20, wait for it to stop rolling, read d20
- Add attack bonus
- “Does a 15 hit?” “Yes”
- Pick up damage dice
- Roll damage dice, wait for them to stop rolling, add damage bonus
- “I deal 12 damage”
- GM adds to monster damage, checks if it’s dead. Play continues.\
Eight steps for the simplest possible "I attack with my sword" turn. Gods forbid you cast a spell, move, or use any kind of special ability. It's hard to simplify beyond three steps of pick up dice, roll dice, update game state, but every additional step adds friction, and as a designer you have to think about if the additional friction is worth the cost. The exception is that big and rare attacks can take time to execute to feel weighty. About once per session per player feels right. Mythic Space does some interesting ease of play work with positive and negative conditions which cancel each other out, represented by blue and red poker chips. Having a condition tracker with stacks of chips and reminders of what they all mean is faster than writing and remembering what the "Dazed" condition does in this game.
The GM is a Player Too
This doesn't fit in the OODA model, but running the game is also playing the game, and all the principles apply above. The difference is that while player characters should be deep and stateful to support mastery and interest over a campaign, the GM is running a constantly shifting set of monsters and NPCs, from balanced squads to hordes of minions and singular bosses. Deep and stateful adversaries lead to cognitive overload for the GM, D&D 3.x being an exceptional example of why this is a disaster in play. D&D 4e monster design is really good, Lancer is okay, but Blood Neon makes an innovation with a cardboard AI for its enemies, so that the GM isn't overloaded with choices about what to do. This also allows players to figure out how the enemies move and attack, and actually get inside their OODA loop with proper positioning.
Generally, enemies should be much simpler than player characters. And because it's exceptionally hard to balance differing numbers of actions per turn, GM activations should be keyed to the number of characters, with the rest of the enemies running on inverse ninja rules of circling threateningly rather than attacking. Blood Neon perhaps goes too far in removing choice, though its enemies are supposed to be mindless, but cardboard AI plus a limited number of specific GM moves strikes a balance.
Are Grids a Good Idea?
Grids are deeply embedded in tactical RPG design as a discrete representation of continuous space. We're spatial apes, and the grid provides a clear and unambiguous representation of the battlespace. Certainly, I don't want to break out the tape measures every turn. But grids also make things static and fiddly, with a lot of square counting and questions about how much moving one square matters. Flying Circus by Erika Chappell is absolutely brilliant in giving players a representation of their plane's energy-maneuverability state (What up, Boyd?) as a paper dashboard with an altimeter, speedometer, engine RPM, G-stress, and ammo, with the maneuvers in the game trading off various dials because you can't map a dogfight, and in one of my favorite lines in any RPG, "Pilots, like babies, lack object permanence." Grids work for a lot of games, but the Flying Circus dashboard model might work for games where grids don't work.