The clip says he doesn't mention Ventura by name in the book.
I think it's entirely believable. Anytime somebody starts buying into asinine government conspiracies, like the government/military/civilians/foreign terrorists working together to kidnap and murder Americans, hide their bodies, shoot missiles into the Pentagon, etc., they're definitely more prone to make the kinds of comments Jesse allegedly made to this guy.