Woman is the moon and man is the sun. But our sexual organs are reversed. So sexual attraction is the desire to be complete by matching the sun to the sun and the moon to the moon. It is also the desire to balance our own energy with it's opposite.
The result can be (but most often isn't) an opening of all five vital channels, allowing prana to flow through the whole body. But for this goal, the journey is much longer, much slower and much more gentle. It requires patience and time and is much less exciting than the way sex is usually perceived in modern life.
When Ayurveda (and many other spiritual paradigms) advocate celibacy, what they certainly do not mean is an obstruction or denial of this natural energy flow. One man said, upon finding a pornographic magazine in a Sadhu's hut "It is better to have a woman under the arm then in the head"
If you have sexual desires you must respond to these natural urges in a responsible and healthy way. What needs to be worked on is the mind, the goal is to no longer desire sex, the goal is not to no longer have sex.