Let's consider how dwarves are portrayed in D&D 4E: Stereotypically short and strong with a penchant for building and craftwork, yet they live above-ground, without any sort of ability to see in the dark, and were once enslaved by the giants. Let's riff off that. Modern dwarves are, in fact, a slave-race bred by giants for manual labor, selecting them for strength, hardiness, and ability to do tedious work (and losing darkvision in the process). They eventually revolted and won their freedom, spreading out into the world.
Where'd they come from, though? The original dwarves were prisoners of war the giants took when they just barely managed to beat back the advances of the great dwarven empire that lies deep beneath the earth. Modern dwarves are what happens when you take these guys and try to break their spirit, and even those guys aren't exactly known for their pliant nature. True dwarves are a fearsome race—not as strong as surface dwarves, but ten times more vicious and cunning. Stealing a bit from Exalted's Mountain Folk, their arts of war have been honed by millennia of conflict with the horrors of the deeps. The forces of the Far Realm, led by Illithid overlords, have been trying to break into the world since time immemorial, and it is only the efforts of the True Dwarves that hold them back. It has been so long since they saw the sun that their race as a whole has forgotten its existence. As far as they know, the only thing that lies above them is the kingdoms of the Drow. Were they to ever engage them seriously, they would quickly break through and discover that beyond lies a lush, weak world ripe for the taking. The surface would face an invasion the likes of which have never been seen, and the True Dwarves would probably be perfectly content to kick their pathetic surface cousins right back down into the role of slaves, along with everyone else they met.