monkeypatch.utils

monkeypatch.utils

Shared utils for the monkeypatches

Functions

Name	Description
get_cu_seqlens	generate a cumulative sequence length mask for flash attention using attn mask
get_cu_seqlens_from_pos_ids	generate a cumulative sequence length mask for flash attention using pos ids

get_cu_seqlens

monkeypatch.utils.get_cu_seqlens(attn_mask)

generate a cumulative sequence length mask for flash attention using attn mask

get_cu_seqlens_from_pos_ids

monkeypatch.utils.get_cu_seqlens_from_pos_ids(position_ids)

generate a cumulative sequence length mask for flash attention using pos ids