Particle collision Physics with RBD

[breadbox]

I'm trying to get good physical interaction between a baked RBD geo and a particle system and the particles seem like they are either all stuck or all slide with nothing in between? 

I'm bringing in the RBD as a static object collider, with deforming geometry turn on, and set the collisions to Use Surface Collisions (the green guide one). The particles collide but for the most part they stick really harshly even when friction is turned down all the way. 

If I go onto the pop solver and in the collision behaviour tab, set the response to slide.. they slide but then it looks like its on ice and the friction controls are disabled.

Any ideas how to get a good setup?  I'm trying to make something like rock debris on a cracking ground surface. so there should be some tumble and roll but eventually come to a stop, maybe a slight bit of bounce but not really too "bouncy" either.