To sing is an infinitive. Infinitives are used for five main reasons which are as the subject of the sentence, as a direct object, subject complement, an adjective or as an adverb. In this case, if the infinitive was being used accurately in the sentence, it would be a direct object. However, Lionel doesn’t hate his own singing, he hates Adam’s singing. This shouldn’t be an infinitive.
The part “his to sing” should be changed to “his singing” because it is grammatically correct and it modifies and describes Adam’s singing. Therefore, a gerund must be used. Using the gerund in the sentence after the verb with a possessive pronoun would be the correct way of expressing the idea of this sentence.