What does "cold comfort" mean?

Slight or no consolation. For example, He can't lend us his canoe but will tell us where to rent one—that's cold comfort. The adjective cold was being applied to comfort in this sense by the early 1300s, and Shakespeare used the idiom numerous times.