You worry too much. When casting lead for bullets (or sinkers) with your Lee casting pot or similiar, you will not be able to get lead anywhere near hot enough to make "lead fumes". You are not going to get lead poisoning from the smell. If your mate got lead poisoning it's because he ingested it. You have to eat lead to get poisoned by it.
I expect cartridge reloading would result in more lead poisoning simply due to the soot on empty cases getting on your hands and then eating lunch. But no one is worrying about that. Not even me to be honest.
Bookmarks