Above: Oscar Wilde in 1882.
From the Quote Investigator website:
…In November 1889 a periodical called “The Theatre” presented the tale together with another remark by Whistler:
Whistler, the eccentric artist that scampers mid-street after auburn-topped models, and that works in blue and silver now and then, is one of the wittiest men in London. At a dinner, the other evening, when Whistler was being applauded for some witty remark that he had let fly, Oscar Wilde, who lives on the bright sayings of others, drawled: “How-I-wish-I-had-said-it!” Whistler turning toward him, exclaimed: “Ah! but you will say it, don’t you know.”
Another time, when one of those absinthe-loaded chappies called on Whistler to tell him that “he and Velasquez were the only artists that ever lived,” Whistler replied to his flatterer with: “Why drag in Velasquez—eh?”…
In 1896 McClure’s Magazine published a profile titled “Whistler, Painter and Comedian” in which the author asserted Whistler had carefully prepared for the delivery of his caustic witticism:
It is impossible to be in Mr. Whistler’s society long without hearing him talk of “getting off” amazing or astounding things, by which he means epigrams. He will spare no pains to get off a good one, and will lead up to it with the most painstaking ingenuity, so that it may at last be jerked out in a natural way. His best-known mot—when Oscar Wilde said to him, “I wish I had said that, Jimmy;” and he said, “Oscar, you will;”—was the result of days of preparation, and was carefully treasured up until the right moment came to “get it off.”
In 1913 “Twenty Years of My Life” by Douglas Sladen was released, and Sladen wrote that he had directly observed the famous incident:
I remember Whistler the painter and Oscar Wilde being the first two people to arrive at a reception at Mrs. Jopling’s house in Beaufort Street, where I had been lunching.
Wilde made a statement that was critical of Whistler that involved traveling a route through Dieppe. Whistler’s later acerbic remark was a form of retaliation according to the author. Sladen’s version of this story was distinctive because he credited an anonymous woman with making the preliminary comment:
It was at this at-home later on that Whistler made his often-quoted mot—not for the first time, I believe. A pretty woman said something clever, and Wilde, who could be a courtier, gallantly remarked that he wished he had said it.
“Never mind, Oscar,” said Whistler, who owed him one for the gibe about the Dieppe route; “you will have said it.”
In 1946 the biography “Oscar Wilde: His Life and Wit” by Hesketh Pearson was published, and Pearson presented an engaging account that included a preliminary remark from Whistler:
Humphry Ward, art critic of The Times, was at an exhibition of Whistler’s paintings, expressing his opinion that one work was good, another bad, and so on.
“My dear fellow,” said Whistler, “you must never say this painting is good or that bad. Good and bad are not terms to be used by you. But you may say ‘I like this’ or ‘I don’t like that,’ and you will be within your rights. Now come and have a whisky: you’re sure to like that.”
“I wish I had said that!” exclaimed Wilde delightedly.
“You will, Oscar, you will,” retorted Whistler with his loud “Ha-ha!”
And there is not the least doubt that Wilde laughed louder than anyone else at this sally, as he usually did when the joke was a good one against himself…”