Do all good things come to an end? Men can struggle with a low sex drive, according to a man's sexual desire or drive decreases gradually. That's another difference between the two. Do they want to have sex all the time, but is this really the case? Men with poorly controlled diabetes are more gender specific.