To my knowledge, the Quake II engine is fairly similar to the Quake engine, with the largest differences (e.g. coloured lighting) having long since been ported back to Quake in some form or another. Tech-wise, I don't think Quake II can provide much that Quake can't these days.
My best attempt at an diplomatic guess is that Quake was just more popular in the long run. There's a possibility that it just hit a critical mass of level designers first, and the gravitational well formed by their output served to attract fresh faces, until it became a self-sustaining feedback loop. When I see other people make cool stuff in an engine, I start itching to do the same. Are there great custom Quake II maps out there? I have no doubt. But they ain't anywhere where I can see them.
I have other theories I could voice, but they're a great deal less impartial and hinge mostly on me not having a very high opinion of Quake II.
