Any resistance in the water slows you down. Hobies can sail with one rudder up.
Generally the rudder that is up is the windward one, it will reduce drag and wont affect steering detrimentally.
Also I have seen rudders up on windward hulls in low wind conditions as well to again reduce drag.