mp.tasks.retrieval.UniversalEmbedder

Class that performs embedding extraction on multi-modal inputs.

mp.tasks.retrieval.UniversalEmbedder(
    lib: serial_dispatcher.SerialDispatcher, handle: ctypes.c_void_p
) -> None

Methods

close

View source

close() -> None

Shuts down and releases the UniversalEmbedder.

cosine_similarity

View source

@classmethod
cosine_similarity(
    u: mp.tasks.components.containers.Embedding,
    v: mp.tasks.components.containers.Embedding
) -> float

Utility function to compute cosine similarity between two embedding entries.

May return an InvalidArgumentError if e.g. the feature vectors are of different types (quantized vs. float), have different sizes, or have a an L2-norm of 0.

Args
u An embedding entry.
v An embedding entry.

Returns
The cosine similarity for the two embeddings.

Raises
ValueError May return an error if e.g. the feature vectors are of different types (quantized vs. float), have different sizes, or have an L2-norm of 0.

create_from_options

View source

@classmethod
create_from_options(
    options: mp.tasks.retrieval.UniversalEmbedderOptions
) -> 'UniversalEmbedder'

Creates the UniversalEmbedder object from options.

embed_audio

View source

embed_audio(
    audio_data: List[float]
) -> mp.tasks.retrieval.UniversalEmbedderResult

Extracts embedding from float audio samples.

embed_image

View source

embed_image(
    image_bytes: bytes
) -> mp.tasks.retrieval.UniversalEmbedderResult

Extracts embedding from raw image bytes.

embed_text

View source

embed_text(
    text: str
) -> mp.tasks.retrieval.UniversalEmbedderResult

Extracts embedding from input text.

__enter__

View source

__enter__() -> 'UniversalEmbedder'

__exit__

View source

__exit__(
    exc_type, exc_val, exc_tb
) -> None