When we can’t talk properly in English at some point, we usually apologise for our bad English/language, and at that time we usually say, “Sorry for my bad English/language.” You can replace it with one idiom, and that is “Pardon my French.” It is the same as saying sorry for my bad English or sorry for offensive English or language.

If you’re wondering why we are using the word French in this context, it’s because it’s one of the most difficult languages for non-native speakers to pronounce. It is easy to learn, but difficult to pronounce. Because of this, whenever you say something, you make a mistake.
When you (non-native speakers) talk to French people, you (non-native speakers) usually use the phrase “excuse my French” when they make mistakes. So a native French speaker then understands that the person (foreigner) has a problem speaking my language.