Thanks, everyone, for your thoughts. I think my limited and second-hand experience revolves around the differences between theory and practice (which, as they say, in theory is the same, but in practice is not).
Here is a specific example, on which I was called by a "Certified Scrum Master". I noticed in a group of classes that some code was doing type-specific functionality in an if/else kind of structure, and observed that it was an opportunity to use the Template Method pattern to allow the subclasses to "do their thing". The overall structure was defined at the superclass level, and each subclass had the ability to perform one or more steps in the algorithm according to their type. It was, to me, a classic case of using a Design Pattern to clean up an obviously bad structure in the first place, and to provide a good basis for future changes...and I think of Design Patterns as kind of "tactical" design tools, as opposed to "strategic" design decisions. I estimated that it would take about an hour to code and maybe half that to test. Anyway, the Scrum Master made it very clear that we should not do anything that was not specifically called for as part of a deliverable, and that the definition of the deliverable was to be viewed as that of functionality, not (as he put it) "nice to haves".
So where does the concept of design changes that DON'T specifically change functionality fall in the agile spectrum? More than one person has told me that the "Certified Scrum Master" was just plain wrong in his judgment on this issue, but honestly I see his position as being right in line with the overall philosophy of agile. My position was simple: the original code was just bad, by almost anyone's measure. And yet it worked, if in a really nasty and hard-coded way. Again, my exposure and experience is quite limited, so please help me understand when and where non-functional design changes fits in Scrum or XP iterations.