Footage published online, which appeared to have been filmed by Louis Tomlinson during their tour in Peru, seemed to show him smoking cannabis with bandmate Zayn Malik.
Some fans of the boyband have criticised them but Cowell spoke in their favour during the Health Lottery fundraising event in London on Monday.
He said: “What I can say is, I’ve worked with the boys for a long time. They are the hardest working boys I’ve ever worked with, they still haven’t changed, they’re really respectful to their fans, they’re hard working, they’re a great British export, but that’s all I can really say.”
Asked if their fans would support them, he said: “I hope the fans stay loyal but that’s all I can really say.
“I’ve worked with a lot of artists in the past who do lose the plot and do lose respect for their fans and forget who put them there – they’ve never done that. They really do love their fans so I can only talk about my own personal relationship with them and they’ve been a joy to work with.”
Simon also said he had “settled” on the fourth judge for the next series of the X Factor. The choice will join Cowell, Cheryl Cole and Louis Walsh on the panel.
He said: “We’ve settled on someone and we’re going to make an announcement this week. It’s not just finding a name, it’s someone who’s going to do the job.
“They’ve got to really commit to the artists and that’s why it’s been a long decision. It’s a female.”