The Hot Wife

So I found myself in an interesting situation. It was a case that went in a convoluted direction and I'll spare you readers the details. But I was seeing this patient who was initially sent to me for evaluation of hypogonadism (low testosterone); he had presented to his primary physician complaining of low libido. He came back to me to review results, this time bringing his wife along for the meeting.
Now, she was a fairly striking woman probably in her late 30's, who dressed well too. As we were talking about the numbers and the effects a patient may experience, Mr. P burst out, obviously frustrated by his symptoms:

"Come on, Dr. V. Look at my wife! Don't you think she's hot? Shouldn't I be more turned on?"

Now, is there ever a proper answer to that question?

(I took the coward's way out and didn't answer the question- just told him treatment should help with the sex drive)