If it's OFF, averaging will act like with any other brush. I think, I might've called this "replace" mode, but I'm not sure... man, my eyes are burning, I'm a little under the weather.
So, that's what confused me about your post.
Because, if it is OFF, it doesn't "pick up color", but just average the colors inside the brush radius, like it always does.
But if it's ON, it's actually a very special mode, as I described above there. I think, I have it then also preserve the material amount properly.