To say All Is One is to immediately posit the opposite, All Is Not One. Because if you are positing All you are positing difference. With difference, the Allness encompasses lack of oneness. It might encompass oneness but that does not negate the multiplicity. You might have oneness contained in the All, but additionally you would have Twoness, Threeness, and so on. In fact, even without the concept of "all," the concept of One is enought to imply the concept of Two. There is no one without a second, especially in the field of time, which is why we call the units of time "seconds."
What about outside the field of time? Cannot there be an Allness that is just God, that knows no separation in its essence? Certainly. But if you are talking about all of the All, you have to include the relative world and its infinite variety. Whatever the world might reduce to in terms of a primordial metaphysical soup does not obviate the distinguishing characteristics of its objective phenomenal differentiation.
Really, it is far more interesting to conceptualize the All as all-encompassing, so that it can include our human experience. It seems illogical of the universe to organize these rich, colorful, variegated lives for us if they did not contribute something to the overall canvas.