Elegance is in the eye of the user.
I state that because the perception of elegance is a personal one, and the best you as a designer can do is define what you think elegance is and then provide a solution that meets that definition. So, here's my definition of what makes software elegant:
It gets out of the way. Elegant software provides an interface that supports the user's primary tasks and doesn't clutter or abstract the tools the user needs to accomplish the task. It is simple, obvious, straightforward and requires very little intellectual effort to understand immediately. It can be a stock tracker, a game, a word processor… Anything, as long as it is designed with a focused intent.
It is designed intelligently and thoughtfully. If there is a hierarchy of controls, the core functions are right up front, and the secondary controls are available at a level beyond the first. I had one of my best design reviews ever when a key stakeholder just absolutely ripped one of my designs apart, because it did not present the controls in a thoughtful way. "it looks like you just threw a Buchanan of buttons on the screen," He said… And he was right. Elegant design is thoughtful design.
It speaks the same language as the user. You can design a screen for an airline pilot that will totally not work for a registered nurse, and vice-versa. Elegant design should be focused on the user and their particular needs, and speaks their language. Design that tries to be all thongs for all people inevitably ends up being bad. understand your user, support their needs, and do it using the terms and conventions that work for them.
One last thought: elegance does not necessarily line up or align with delight. The emotion of delight can be triggered by many many things, and you can provide an elegant solution that does not provoke delight. If anything, the ability to provoke delight is getting harder, as the baseline of user expectations continue to rise as they are exposed to ever more refined technology. Delight is hard, and elegance can provoke delight… Sometimes.