One of the science-fiction magazines I used to read was called Worlds of If. How apropos a title. The word "if" heralds untold possibilities for alternate worlds. Consider it the jumping-off place for parallel universes, which, if certain physicists are to be believed, are constantly branching off from this reality. Conceived as such, reality is very open-ended and in fact continuously generating different versions of itself at any given moment.
Were we to follow any of these branches, we would likely soon get lost and never find our way back to our own reality. But there is much information about these parallel worlds we can glean from the facts that are in front of us, that makes it somewhat unnecessary to actually visit them. We can know alternate realities through holographic research into our own minds. That is, by the art of "delving," or looking into the mind concerning a certain point we are curious about, we will receive definitive information about that thing. This process takes advantage of the holographic structure of reality whereby all potential knowledge is embedded in the thing itself and the seeds of all things are embedded in our minds.
Once one has become adept at delving, one can then explore parallel universes at will. The multiverse is indeed an infinite library such as Borges wrote about.