When you are charged with a crime, you have one of two pretty standard defenses. The first is that you are innocent of the crime. The second is that you did do the crime but for one reason or another, you should not be held accountable in a criminal sense.
If you didn't commit the crime for which you are being charged, the defense may seem simple. You now are under no obligation to prove you didn't do it, but doing this is beneficial to your case. The prosecution is simultaneously looking to procure any evidence to the contrary.
Since law decrees any person charged with a crime is to be presumed innocent until prosecutors prove their guilt, you can, in fact, plead the fifth and refuse to answer any questions, thereby making it clear the prosecution cannot use any of your own words against you in trying to prove guilt. Your defense here is key because while the responsibility is not on you to prove your innocence, demonstrating that there could be a chance you did not do what you're charged with should effectively result in your being found innocent.
If you have a substantiated alibi, you may be able to effectively resolve the burden of proof, and see charges against you do not stick.
It is important to act quickly upon your arrest for a crime. Your prosecution is actively attempting to find you guilty, and while you may benefit from remaining quiet and noncompliant with interrogation, without the counsel of an attorney, you may still go to work to demonstrate the probability of your innocence.