What goes wrong when impediments are not removed? Let me put it simply: the team can not succeed! Our team has gone sprint after sprint after sprint saying the *same* things over and over. Here's an example: we use TFS for source control. The box that houses TFS is DOG SLOW .... for those who use the web interface, you click a link, and then go get a cup of coffee ... Every sprint for 9 months, the team has said "we need a better TFS box".
When we didn't get one, it didn't take a lot of time before people resisted using it. And when some people stopped using it (for tracking things like Product Backlog Items and Sprint Backlog Items!!), *that* behavior became an impediment to other team members. Over a few sprints, we eventually dropped the electronic tracking of user story cards altogether!
Back to index cards we went ... but *sigh*. Rather than work toward solving the original problem (faster TFS box!), we started trying to solve the problems that we had with index cards. We have remote employees, and they couldn't see the cards. Worse, the cards lacked detail, and we never have yet solved the problem of several team members not knowing more than just a few words of information about a card.
Our team has spent sprint after sprint trying to work around problems that we have previously identified but not been able to solve, so much so that we are losing sight of our original problems, and are starting to work around problems that some of us have forgotten about.