It's hard to really control the force input by a hammer when you're hitting something relatively springy and not firmly located. This is why a blacksmith uses an anvil.
A forged wheel will be relatively resistant to bending, but the grain structure of the metal lends itself to being bent back. A cast wheel is fundamentally brittle and is much more likely to crack.
Were I to attempt this on a cast wheel I'd demount the tire and make a wooden block shaped to match the interior of the bead seat out of hard maple or at least oak. I'd use a bandsaw and a round-over bit in a router. Ideally the block would be angled on the bottom so the wheel, when resting in the block which in turn would be atop my anvil - or a curb, or a CMU on a concrete floor - would "present" the damaged area for easy access.
I'd put a few plies of thick, old school masking tape on the damaged area and use a dead blow hammer with a hard plastic face, or my BFLRHH - "Big berkeleying Lead and Raw Hide Hammer" and investigate precisely how hard it needs to be hit for the metal to move.
Work GRADUALLY back and forth along the area and take your time. One good wallop and a cast wheel lip will crack. Work it gently and it may go back.
I wouldn't use that wheel on track afterwards even if you're "OK' with it. It's not fair to everyone else.