nebius.base.token_sanitizer.TokenVersion

class documentation

class TokenVersion: (source)

Constructor: TokenVersion(prefix, delimiter, signature_position, token_parts_count)

View In Hierarchy

Describe the structure of a token version.

The structure contains a prefix, a delimiter, a signature position, and an expected number of token parts.

Parameters
prefixThe prefix that identifies this token version.
delimiterThe delimiter used to split the token into parts.
signature_positionThe zero-based index of the signature part.
token_parts_countThe expected number of parts.
Method __init__ Undocumented
Instance Variable delimiter The delimiter used to split the token into parts.
Instance Variable prefix The prefix that identifies this token version.
Instance Variable signature_position The zero-based index of the signature part in the token.
Instance Variable token_parts_count The expected number of parts when the token is split by delimiter.
def __init__(self, prefix: str, delimiter: str, signature_position: int, token_parts_count: int): (source)

Undocumented

delimiter: str = (source)

The delimiter used to split the token into parts.

prefix: str = (source)

The prefix that identifies this token version.

signature_position: int = (source)

The zero-based index of the signature part in the token.

token_parts_count: int = (source)

The expected number of parts when the token is split by delimiter.