Base

This module contains functionality related to the the base module for embedding.embedders.

Base

BaseEmbedder

Bases: ABC

Abstract base class for text node embedding operations.

Defines interface for embedding generation and storage operations on text nodes.

Attributes:
  • embedding_model

    Model for generating embeddings

  • vector_store

    Storage for embedding vectors

Source code in src/embedding/embedders/base.py
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
class BaseEmbedder(ABC):
    """Abstract base class for text node embedding operations.

    Defines interface for embedding generation and storage operations
    on text nodes.

    Attributes:
        embedding_model: Model for generating embeddings
        vector_store: Storage for embedding vectors
    """

    def __init__(
        self, embedding_model: BaseEmbedding, vector_store: VectorStore
    ):
        """Initialize embedder with model and storage.

        Args:
            embedding_model: Model to generate embeddings
            vector_store: Storage for embedding vectors
        """
        super().__init__()
        self.embedding_model = embedding_model
        self.vector_store = vector_store

    @abstractmethod
    def save(self, nodes: List[TextNode]) -> None:
        """Save embedded text nodes to vector store.

        Args:
            nodes: Collection of text nodes with embeddings
        """
        pass

    @abstractmethod
    def embed(self, nodes: List[TextNode]) -> None:
        """Generate embeddings for text nodes in batches.

        Args:
            nodes: Collection of text nodes to embed

        Note:
            Modifies nodes in-place by adding embeddings
        """
        pass

__init__(embedding_model, vector_store)

Initialize embedder with model and storage.

Parameters:
  • embedding_model (BaseEmbedding) –

    Model to generate embeddings

  • vector_store (VectorStore) –

    Storage for embedding vectors

Source code in src/embedding/embedders/base.py
20
21
22
23
24
25
26
27
28
29
30
31
def __init__(
    self, embedding_model: BaseEmbedding, vector_store: VectorStore
):
    """Initialize embedder with model and storage.

    Args:
        embedding_model: Model to generate embeddings
        vector_store: Storage for embedding vectors
    """
    super().__init__()
    self.embedding_model = embedding_model
    self.vector_store = vector_store

embed(nodes) abstractmethod

Generate embeddings for text nodes in batches.

Parameters:
  • nodes (List[TextNode]) –

    Collection of text nodes to embed

Note

Modifies nodes in-place by adding embeddings

Source code in src/embedding/embedders/base.py
42
43
44
45
46
47
48
49
50
51
52
@abstractmethod
def embed(self, nodes: List[TextNode]) -> None:
    """Generate embeddings for text nodes in batches.

    Args:
        nodes: Collection of text nodes to embed

    Note:
        Modifies nodes in-place by adding embeddings
    """
    pass

save(nodes) abstractmethod

Save embedded text nodes to vector store.

Parameters:
  • nodes (List[TextNode]) –

    Collection of text nodes with embeddings

Source code in src/embedding/embedders/base.py
33
34
35
36
37
38
39
40
@abstractmethod
def save(self, nodes: List[TextNode]) -> None:
    """Save embedded text nodes to vector store.

    Args:
        nodes: Collection of text nodes with embeddings
    """
    pass