Joe, however, fired back in an interview with RadarOnline, saying Kris Jenner is "no Cinderella...[she] is very negative...she's an evil b*tch," and had also told the site that Kris care about Lamar, she just "cares about their image and money."
In an earlier interview, he said, "I wish he had never got married to Khloe to be honest...She's controlling. Those Kardashian women all are. You get married they think they wear the pants. You just bring them your whole paycheck, that's how it works. I saw her totally differently when I heard the way she talked privately--it isn't pleasant."
Now, however, Joe seems to be changing his tune a little bit, telling TMZ that he wasn't really trying to talk bad about them, he just got upset, and said he thinks the family is "getting a bad rap" and apologizes for his comments.
He also claims he is clean and denied that Lamar had a drug problem.
Watch the video above to hear Joe's comments and tell us what you think.