Interface SpeculativeDecodingSpec.NgramSpeculationOrBuilder (1.48.0)

1.48.0 (latest) 1.47.0 1.45.0 1.43.0 1.42.0 1.40.0 1.38.0 1.36.0 1.35.0 1.34.0 1.33.0 1.32.0 1.30.0 1.28.0 1.27.0 1.24.0 1.23.0 1.22.0 1.20.1 1.19.0 1.18.0 1.17.0 1.16.0 1.15.0 1.14.0 1.13.0 1.12.0 1.11.0 1.9.0 1.8.0 1.7.0 1.6.0 1.5.0 1.4.0 1.3.0 1.2.0 1.1.0 1.0.0 0.8.0 0.6.0 0.5.0 0.4.0 0.3.0 0.2.0 0.1.0

public static interface SpeculativeDecodingSpec.NgramSpeculationOrBuilder extends MessageOrBuilder

Implements

MessageOrBuilder

Methods

getNgramSize()

public abstract int getNgramSize()

The number of last N input tokens used as ngram to search/match against the previous prompt sequence. This is equal to the N in N-Gram. The default value is 3 if not specified.

int32 ngram_size = 1;

Returns
Type	Description
int	The ngramSize.