From a technical perspective, they're a really cool combination of a declarative programming model—which allows everything that depends on a given piece of data to know relatively efficiently when and how to recompute itself when the underlying data changes—and an accessible, visual model that makes them usable for people with almost no programming experience at all. The actual formula syntax is more arcane than it really could be for legacy reasons, but that aside it's a really great on-ramp from zero programming experience to making relatively complex interactive applications. I really love tools that make that kind of UI prototyping easy and accessible.
From an aesthetic perspective, spreadsheets represent an ordering of the complex reality of the world into a comprehensible and cogent schema while also being flexible and mutable enough to allow creative reorganization and admixture of that schema as new needs arise. They're the inflection point between understanding and exploration. They're an attempt to recreate the world in microcosm without denying the fact that that's ultimately impossible to do.
i love an excuse to be reminded of + post this wonderful observation about spreadsheets as a user interface. image transcription under the fold, emphasis added by me.