Yeah, I think I got it. Not as complicated like you said. I wanted to somehow still drive the geometry using the alembic cache and slowly transition into the RBD sim, was going look nicer that way. Right now I just write out the frame I want to shatter, read it back in and activate the pieces with activevalue, - not ideal but it works. The rest of the geometry is deformed with cloth capture/deform.
Thanks for the tip!