I think I may have found the issue (I don't know that I fully understand what happened) but I want to add my thoughts in case a future person finds this and needs a place to start to debug a similar issue. Somehow when I imported my allembic file I noticed that in the "animation" section of the object properties the default frame showed as 1.25 (the frame is greyed out but I thought that odd). It was as if somehow it was referencing a quarter frame and I know typically allembics came in with that "frame" settings greyed out (but set to 1F). So I set the speed of my animation to 125% (since 1.25/125% equals 1) and this solved my problem. It was as it every 3-4 frames c4d didn't have animation data to read (or just read the data from the previous frame) causing that pause. Just changing the speed of my animation solved the issue. Its important to note that it needed to be 125 exactly (in my case) as I randomly tried 101% speed just to see if it needed to be any speed except 100 and as expected that didn't work. So while I don't have the complete answer there does seem to be some mathematical logic behind the issue I was seeing.
Speeding up the animation solved it. Though I exported at 24fps into a c4d project that was at 24fps, there may be something related to me using the retime sop in houdini to slow down my animation. I'll post if I get some time to investigate further in case its helpful to somebody else. At least I was able to get the animation working smoothly in C4D, but certainly i'm open to any thoughts that might help me better understand what i'm doing wrong to cause this so I can avoid it entirely in the future.