There's a phrase for that. It's called "Killing your darlings."

My advice would be to write it as it comes when you're rough drafting. By all means, when you're outlining, jot down bits of dialog, scenes, and random sentences! It won't hurt you to write down ideas when they come to you. I do a fair amount of that; if I'm outlining a book and think of a way a particular scene would go, I absolutely sketch it out. This is especially true of dialog. Sometimes I only need to write down a few lines to get the gist, but I capture the idea, because I'll most assuredly lose it otherwise. I almost always end up rewording it when I work it into the book, but it helps me because I have a framework for how I wanted the scene to go. Otherwise I'd be like, "Okay, they were supposed to argue here... Um... What's he gonna say?"
It won't hurt you to write down gems when you think of them, and it won't hurt you to go into a scene hoping to work a certain element in. If that element captures the heart of what you want to convey, build on it! Some people build whole books around the tiniest nugget of inspiration, which could be a short witty line.
The key is to be willing to cut. As the book continues to develop, let it grow as it will. Trust me, you'll know what needs to be cut when you get there. Just let the book manifest itself, and you'll figure out what fits and what doesn't. The finished product will have plenty of beautiful gems all its own.
I went through a lot of this with
Peter's Angel. I could publish a whole 'nother book of darlings I had to cut, from little turns of phrase to entire plotlines. Did I love them? Yeah, but I love the finished book, too. New things emerged out of the revised plot that I love just as well, and I know the book is better--exponentially better--than it was before, as anyone who read multiple drafts will heartily testify.

(And I don't think they miss all the little nuggets of narration they raved over in the rough draft which I subsequently cut, if they even noticed they were gone.)