Bugs make you strong.
It sounds counter-intuitive but every bug you find is one step close to making your code base stronger and more reliable. Naturally, we try to avoid any unpredictability in our code almost as if we want to snip all the loose edge and hide away the frills. But this only postpones the inevitable: the more we postpone fixing issues the larger the break will be. Eventually, the level of entropy in the code is so large that it's not worth fixing at all.
In my younger days of working extensively with code I used to cringe at the thought of my work breaking in unpredictable ways. Then I began to ask myself what the gain would be if it ran today only to crash in a very unpredictable way in a week from now. Nowadays, I pray that it crashes and exposes its wicked side.
Having said that you're better off not having any bugs if you are not committed to ensuring the sanctity of your work. If you don't incorporate tests right from the beginning then you're playing with fire. If you are not committed to return for a refactor and integrate step then you're playing with fire. And if you know that some poor bloke will eventually be responsible and you are on your way to retirement then... move on. But if you want the time-to-feature to be as small as is practically possible then you want every bug to rear its ugly face.