What is the length of the prompt considered by BLOOM?
#236
by akratz - opened
See subject line. How long can a prompt be to be considered in its entirety? What happens if it exceeds some length, is the beginning cut off?
julien-cBLOOM was trained with sequences of length 2048, but uses ALiBi position embeddings (https://arxiv.org/abs/2108.12409) -- meaning it can be used with longer sequences.
So ALIBI allows input of what length to be considered? You write that it allows for “longer sequences” but this way I only know it is more than 2048…
Any update on this? @cakiki
I'd refer you to both the paper I linked to and empirical experimentation to answer that question. The practical limit will likely sooner be your hardware than any theoretical one.
christopher changed discussion status to closed